1 April 2013

Long case clock by John Brown of Kilmarnock featuring Robert Burns.


Burns was friends with a clockmaker called John "Clockie" Brown who made clocks in Mauchline before 1800. This clock is later, dating from 1860, so it's a different John Brown.

All

 

top 

 

Top Left 

 

Top Right 

 

Bottom Right v2 

 

Bottom Left

9 October 2011

Reading CD-Text with VB.Net (3)

Last time we had retrieved a big blob of data that holds the CD-Text. The mmc3 document tells us the blob is a series of CD-TEXT Pack Data, which we can call packs for short. All the information is contained in these packs, but the packs themselves describe a more complicated data structure that we need to recreate. I used the mmc document, patent application and open source projects to understand this structure.

All the text is stored in one Text Group, which is what we have retrieved. The text group contains up to 8 Blocks numbered 0 to 7. It is possible for a CD to have text in multiple languages, each Block contains text in one language only. Each Block is made up of Packs. A block can contain up to 255 Packs, each pack in a block will have a unique Sequence Number 0-255. If the text is particularly large then it may need multiple blocks to fit it all in. (One block contains text for one language only, one language may require multiple blocks.)

Each pack is 18 bytes in size and has three sections – a 4 byte header, 12 bytes of data and 2 bytes for a CRC. The header of a pack contains it’s block number. The header has one byte to hold the sequence number. The header also contains a number that identifies the type of data that the pack stores. For example the pack type &H80 means the data portion of the pack contains text data forming part of the album or track titles. As it’s only 12 bytes of data in a pack, you might need several packs to describe, say, the title of track 3. The track number is also stored in the header, track number 0 is special and means the data relates to the CD as a whole rather than a particular track. For the Title pack type, for example, track number 0 is the title of the album.

An important pack type is the size pack type. Each block has 3 size packs. These do not contain text data, instead the bytes map to various counts and other numerical information. The size packs describe a block’s:

  • character code - how the text data in the pack types that store text is encoded. E.g. ASCII
  • first track number, last track number
  • copy protection info
  • number of packs of each pack type in the block
  • last sequence number of each block (0 if the block number is not used, this information must be repeated in each block as it’s not specific to the block that contains the size packs)
  • language code – this tells us which language the block is for e.g. English, Afrikaans, French etc.

Most of the packs contain text information for the tracks or album. If we have an imaginary CD which has 3 tracks with title data for each track and the album, e.g:

  • Album title = “Hello World Album” (Track 0)
  • Track 1 = “I’m the urban spaceman”
  • Track 2 = “The intro and the outro”
  • Track 3 = “Poisoning pigeons in the park”

Then we would split each string up into chars and append a Chr(0) to indicate the end of that string. We chop it into 12 byte sections and make a pack for each section. This table shows the 18 bytes of a pack, 4 (H)eader bytes, 12 (D)ata bytes and 2 (C)rc bytes. /0 indicates a Chr(0). The first header byte is the pack type – for us &H80. The second header byte contains 7 bits identifying the track number (there’s another bit which we are going to say isn’t set). As the data might contain sections of information from multiple tracks, the track number identifies which track the first byte of track data belongs to. The block sequence number is in the third header byte – let’s imagine that these packs happened to start at block sequence number &H10. The eight bits of information in the fourth header byte encode three things – whether the text uses 1 byte or two per character, the block number (0 – 7) and the position that the first character in the data takes in it’s string. The character position isn’t useful to us (maybe it is useful in C or ASM). Let’s ignore this byte, and the Crc bytes, and just assume it’s all in one block.

H H H H D D D D D D D D D D D D C C
80 0 10 H e l l o   W o r l d  
80 0 11 A l b u m /0 I m   t h
80 1 12 e   u r b a n   s p a c
80 1 13 e m a n /0 T h e   i n t
80 2 14 r o   a n d   t h e   o
80 2 15 u t r o /0 P o i s o n i
80 3 16 n g   p i g e o n s   i
80 3 17 n   t h e   p a r k /0 /0

There are various pack types like this – for the title, performer, songwriter, composer, arranger, messages. There are also other pack types which contain other types of information, like the size pack type. For example, there are pack types for CD Information, Toc information and Genre information. These are always encoded in ASCII, and they are not encoded individually for each language on a multi-language CD.

I imagine that multi-language CDs don’t exist in the wild. When testing burning apps I found none of them would use multiple blocks either. It seems most CDs then will just have 1 block, 1 language and use ASCII, which makes it pretty easy to decode a CD. I’m not exactly sure what happens if you have to split a language over blocks. I guess it just picks up where the last one left off.

In .Net we can have some class CdText to store the properties that apply to the cd as a whole. Then it’s probably easiest to shove the language based information into a datatable. Here we have a primary key of (Language, Track No), and columns for things like Title, Performer, … etc. We can stick some helper properties onto the main class to list the available languages etc.

17 September 2011

Reading CD-Text with VB.Net (2)

I mentioned, in the first part, that there are two control codes that you can send to get the CD-Text bytes back. We’ve used IOCTL_CDROM_READ_TOC_EX and now it’s the turn of the slightly trickier IOCTL_SCSI_PASS_THROUGH_DIRECT. This one lets us send any message we like to the device. To learn about the messages that you can send, you need to get your hands on a copy of the standard specification that defines them all. Hardware manufacturers are supposed to conform to the standard so that all the devices understand the same commands.

The standards of interest are the ones for scsi devices. You have to pay for access to the latest standards, and it looks like access to the ftp for draft standards is now restricted. The old standards are available on the internet in various places though. I still have my copies of mmc3r10g.pdf and spc2r20.pdf. It has some information that will come in handy later about the format of the bytes we have been retrieving. You can find yet more information on this format by looking at open source projects such as cdrdao, the original patent application by Sony and various other projects. The real standard is the Red Book (IEC 60908), but that costs £200 to access.

Anyway, we have to call DeviceIoControl with this other command code. We send it a structure that will contain a cdb (command data block) along with a buffer to contain the bytes that are coming back, and another buffer for so called sense-information. Sense information is helpful as it contains error information – and there are many things that can go wrong – tray out, no cd, DVD inserted, drive not ready, etc. This is probably the only reason to use this message over the other one – you can get back the sense information. It’s also useful to know how to send any command as there are a lot of them in the specification, and they don’t all have an equivalent “friendly” IOCTL code.

The command we are sending is the READ TOC command, the cdb is just 10 bytes in size. It contains one byte for the command code (&H43). Then a byte we can ignore. Then there’s a byte that holds the format. This is the same thing as the format we send with IOCTL_READ_TOC_EX – we can specify different values to get different TOC information. Again, we want to set this to 5 to retrieve CD-Text (see the spec). After that there are reserved, and inapplicable bytes (given the format). At the end there are two bytes to say how big our buffer is, and finally a control byte.

The cdb and a pointer to the databuffer go into a SCSI_PASS_THROUGH_DIRECT structure. This structure is then placed inside another structure that also holds the sense buffer. SCSI_PASS_THROUGH_DIRECT looks like this:

<StructLayout(LayoutKind.Sequential)>
Friend Structure SCSI_PASS_THROUGH_DIRECT                           ' x86   x64
    Public Length As Short                                          '  0      0
    Public ScsiStatus As Byte                                       '  2      2
    Public PathId As Byte                                           '  3      3
    Public TargetId As Byte                                         '  4      4
    Public Lun As Byte                                              '  5      5
    Public CdbLength As Byte                                        '  6      6
    Public SenseInfoLength As Byte                                  '  7      7
    Public DataIn As CommandDirection                               '  8      8
    Public DataTransferLength As Integer                            ' 12     12 
    Public TimeOutValue As Integer                                  ' 16     16
    Public DataBuffer As IntPtr                                     ' 20     24
    Public SenseInfoOffset As Integer                               ' 24     32
    <MarshalAs(UnmanagedType.ByValArray, SizeConst:=16)>
    Public Cdb() As Byte                                            ' 28     36
    '                                                        Size     44     56
    Public Shared Function GetSize() As Integer
        Return Marshal.SizeOf(GetType(SCSI_PASS_THROUGH_DIRECT))
    End Function
 
End Structure

I’ve got the offsets/sizes that I calculated in C in the comments. You can see that the IntPtr field is larger on x64. Here the cdb array is marshalled as a ByValArray which tells the marshaller to stick the bytes directly into the memory of the structure. The alternative is for the bytes to live elsewhere and for the field in the structure to be a pointer to them. We have to fill in this structure, which means we have to know what the PathId, TargetId and Lun of the CD drive are. We can get those with IOCTL_SCSI_GET_ADDRESS. The SenseInfoLength is 32. The SenseInfoOffset can be calculated at runtime using Marshal.OffsetOf to get the offset of the sense buffer in the outer structure (see attached project) – it will vary on x86 and x64.

As before, we’ll call once with a data buffer of size 2. On return, our data buffer will contain the size of the data (excluding the field that holds the size itself). So we can then allocate precisely as much memory as is required to hold the whole CD-Text.

Option Strict On
Option Explicit On
 
Imports CdTextIoCtl.NativeMethods
Imports Microsoft.Win32.SafeHandles
Imports System.ComponentModel
Imports System.IO
Imports System.Runtime.InteropServices
Imports System.Text
 
Public Class CdTextRetriever
 
    Public Shared Function GetCdText(driveInfo As DriveInfo) As Object
        Dim cdText As Object = Nothing ' don't know what the return will look like yet.
        Dim devicePath As String = String.Format("\\.\{0}:", driveInfo.Name(0))
        Const FILE_ATTRIBUTE_NORMAL As UInteger = &H80
        Const GENERIC_READ As UInteger = &H80000000UI
        Const GENERIC_WRITE As UInteger = &H40000000
        Const FILE_SHARE_READ As UInteger = 1
        Const FILE_SHARE_WRITE As UInteger = 2
        Const OPEN_EXISTING As UInteger = 3
        Using hDevice As SafeFileHandle = NativeMethods.CreateFile(devicePath,
                GENERIC_READ Or GENERIC_WRITE,
                FILE_SHARE_READ Or FILE_SHARE_WRITE,
                IntPtr.Zero, OPEN_EXISTING,
                FILE_ATTRIBUTE_NORMAL, IntPtr.Zero)
            If hDevice.IsInvalid OrElse hDevice.IsClosed Then Throw New Win32Exception
            ReadCdText(hDevice)
        End Using
        Return cdText
    End Function
 
 
    Private Shared Sub ReadCdText(hDevice As SafeFileHandle)
        Dim address As SCSI_ADDRESS = GetScsiAddress(hDevice)
        ' like in the other program...get the length
        Dim bytes(1) As Byte
        GetCdTextBytes(hDevice, address, bytes)
        ' again we add 2 for the length parameter itself so it's length - 1 + 2 to size the array
        bytes = New Byte((CType(bytes(0), Integer) << 8) Or bytes(1) + 1) {}
        GetCdTextBytes(hDevice, address, bytes)
        Dim sb As New StringBuilder
        For i As Integer = 4 To bytes.Length - 19 Step 18
            For j As Integer = 0 To 17
                sb.Append(bytes(i + j).ToString("X2"))
                If i = 3 OrElse i = 16 Then sb.Append("  ")
            Next
            sb.AppendLine()
        Next
        My.Computer.FileSystem.WriteAllText(IO.Path.Combine(My.Computer.FileSystem.SpecialDirectories.MyDocuments, IO.Path.GetRandomFileName), sb.ToString, False)
    End Sub
 
    Private Shared Function GetScsiAddress(hDevice As SafeFileHandle) As SCSI_ADDRESS
        Dim address As New SCSI_ADDRESS
        Dim bytesReturned As Integer
        Const IOCTL_SCSI_GET_ADDRESS As Integer = &H41018
        Dim result As Boolean = NativeMethods.DeviceIoControl(hDevice,
                IOCTL_SCSI_GET_ADDRESS, IntPtr.Zero, 0, address,
                SCSI_ADDRESS.GetSize, bytesReturned, IntPtr.Zero)
        If result = False Then Throw New Win32Exception
    End Function
 
    Private Shared Sub GetCdTextBytes(hDevice As SafeFileHandle, address As SCSI_ADDRESS, bytes() As Byte)
        Dim bytesReturned As Integer
        Dim sptdwb As New SCSI_PASS_THROUGH_DIRECT_WITH_BUFFER
        Dim pinnedBytesHandle As GCHandle = GCHandle.Alloc(bytes, GCHandleType.Pinned)
        sptdwb.ScsiPassThroughDirect.cdb = New Byte(15) {} ' The cdb as described in the standard
        sptdwb.SenseBuffer = New Byte(31) {}
        sptdwb.ScsiPassThroughDirect.Length = CShort(SCSI_PASS_THROUGH_DIRECT.GetSize)
        sptdwb.ScsiPassThroughDirect.CdbLength = 10
        sptdwb.ScsiPassThroughDirect.DataIn = CommandDirection.SCSI_IOCTL_DATA_IN
        sptdwb.ScsiPassThroughDirect.DataTransferLength = bytes.Length
        sptdwb.ScsiPassThroughDirect.DataBuffer = pinnedBytesHandle.AddrOfPinnedObject
        sptdwb.ScsiPassThroughDirect.TimeOutValue = 10
        sptdwb.ScsiPassThroughDirect.SenseInfoOffset = Marshal.OffsetOf(GetType(SCSI_PASS_THROUGH_DIRECT_WITH_BUFFER), "SenseBuffer").ToInt32
        sptdwb.ScsiPassThroughDirect.SenseInfoLength = 24
 
        sptdwb.ScsiPassThroughDirect.PathId = address.PathId
        sptdwb.ScsiPassThroughDirect.TargetId = address.TargetId
        sptdwb.ScsiPassThroughDirect.Lun = address.Lun
        ' CDB is set with the READ_TOC command
        sptdwb.ScsiPassThroughDirect.cdb(0) = &H43
        sptdwb.ScsiPassThroughDirect.cdb(2) = 5
        sptdwb.ScsiPassThroughDirect.cdb(7) = CByte((bytes.Length >> 8) And &HFF)
        sptdwb.ScsiPassThroughDirect.cdb(8) = CByte(bytes.Length And &HFF)
        Try
            Const IOCTL_SCSI_PASS_THROUGH_DIRECT As Integer = &H4D014
            Dim result As Boolean = NativeMethods.DeviceIoControl(hDevice,
               IOCTL_SCSI_PASS_THROUGH_DIRECT,
               sptdwb,
               Marshal.SizeOf(sptdwb),
               sptdwb,
               Marshal.SizeOf(sptdwb),
               bytesReturned,
               IntPtr.Zero)
            If result = False Then Throw New Win32Exception
        Finally
            pinnedBytesHandle.Free()
        End Try
    End Sub
 
End Class

So, that’s another way to read the same bytes. I’ve told this one to save a file with a dump of the bytes into MyDocuments. It splits the data into lines with 18 bytes on each. It starts at byte 4 to skip the length and two other bytes which appear before the Pack Data. Pack Data contains 12 bytes of information each with a header describing the information and a 2 byte CRC code at the end. Most of the information will be ascii text, although some packs don’t contain text. The header contains a pack type code which has things like Title, Songwriter, etc. It also has values to identify the track number that the data is for. It’s quite a simple format, but tricky to decode without a precise definition of the format! These projects are a bit naff – I’ve not used VS2010 much and it’s messing up my solution with 2 projects in. I can’t seem to get it so that you can switch easily between x86 and AnyCpu builds, so it ends up in mixed mode. You can just start a new empty solution and add the relevant bits if it wont build and run.

Next time I’ll decode the bytes, check the crcs and the sense information. I’ve found that occasionally I get some corrupt data coming in the first time it reads – hopefully the CRC will fail and I can retry.

11 September 2011

Reading CD-Text with VB.Net (1)

Note: most commercial CDs do not have any CD Text on them.

There are probably a few ways to read the CD Text. I’m going to look at using the windows api function DeviceIOControl function. The steps are (1) Get a device handle (2) Prepare a message (3) Send the message to the device (4) Decode the results.

To get a device handle, you call CreateFile sending the drive letter of the drive formatted as a device path. You also have to specify the minimum permissions you will be needing for the operations you will be performing. If you fail to ask for a handle with sufficient permissions then you will get errors later on when you are calling DeviceIOControl.

In .Net you can pick various types to represent handles. I’ve seen people use Integer – which isn’t good as Integer is always 32 bits whereas handles may be 64 bits. IntPtr is a better choice as it will match the size of the C HANDLE type it is representing. HandleRef is another good choice, this is used to prevent the garbage collector from destroying your handle before a call ends – which would be embarrassing. SafeHandle is the best choice to use as it ensures that the handle will be closed in certain unusual circumstances. SafeHandle is an abstract class, there are various concrete implementations. For a device handle we can use the SafeFileHandle.

The IOCTL_CDROM_READ_TOC_EX control code can be used to get the CD-Text back from the drive. Another alternative is to use a less specific control code such as  IOCTL_SCSI_PASS_THROUGH which lets you send any command to the drive, it’s a bit more involved – I’ll show how to use that later. When you are working from scratch, and trying to figure this stuff out for yourself, it helps to have a C++ project that you can use to dump out the values of the constants, size of structures, and offsets of structure fields. A bit of work in C++ can save a lot of time searching the internet with vague error messages like “A call to PInvoke function xyz has unbalanced the stack”.

If you want to set up a C++ project then the steps are:
1) Download and install the WDK. It’s a big download. The bit you need are the “Build Environments”, as selecting this will ensure that the header files are installed.
2) Start visual studio 2010 and a new C++ CLR Console application project called, say, StructSizes
3) Right click the project name in the solution explorer, click References, Add New Reference and add System.Windows.Forms.
4) Right click the project name in the solution explorer, click Properties, Drill down to Configuration Properties/VC++ Directories and then select Include Directories. Edit the string to include the path to the wdk header files. This will involve adding a “;” and then the path. Mine looks like this after the edit: $(VCInstallDir)include;$(VCInstallDir)atlmfc\include;$(WindowsSdkDir)include;$(FrameworkSDKDir)\include;C:\WinDDK\7600.16385.1\inc\api
5) As an example lets dump out the value of the IOCTL command and some info about a structure. This goes into the cpp file with the same name as the project, so StructSizes.cpp.

#include "stdafx.h"
#include <Windows.h>
#include <winioctl.h>
#include <ntddcdrm.h>
#include <ntddscsi.h>
#include <stddef.h> 
 
using namespace System;
using namespace System::Text;
using namespace System::Windows::Forms;
 
[STAThread()] // to make the clipboard happy.
int main(array<System::String ^> ^args)
{
    StringBuilder ^sb = gcnew System::Text::StringBuilder();
    sb->AppendLine(L"IOCTL_CDROM_READ_TOC_EX: " + (IOCTL_CDROM_READ_TOC_EX).ToString(L"x2"));
    sb->AppendLine(L"SCSI_PASS_THROUGH offsets + size:");    
    sb->AppendLine((offsetof(SCSI_PASS_THROUGH, SCSI_PASS_THROUGH::Length)).ToString() + L" Length");
    sb->AppendLine((offsetof(SCSI_PASS_THROUGH, SCSI_PASS_THROUGH::ScsiStatus)).ToString() + L" ScsiStatus");
    sb->AppendLine((offsetof(SCSI_PASS_THROUGH, SCSI_PASS_THROUGH::PathId)).ToString() + L" PathId");
    sb->AppendLine((offsetof(SCSI_PASS_THROUGH, SCSI_PASS_THROUGH::TargetId)).ToString() + L" TargetId");
    sb->AppendLine((offsetof(SCSI_PASS_THROUGH, SCSI_PASS_THROUGH::Lun)).ToString() + L" Lun");
    sb->AppendLine((offsetof(SCSI_PASS_THROUGH, SCSI_PASS_THROUGH::CdbLength)).ToString() + L" CdbLength");
    sb->AppendLine((offsetof(SCSI_PASS_THROUGH, SCSI_PASS_THROUGH::SenseInfoLength)).ToString() + L" SenseInfoLength");
    sb->AppendLine((offsetof(SCSI_PASS_THROUGH, SCSI_PASS_THROUGH::DataIn)).ToString() + L" DataIn");
    sb->AppendLine((offsetof(SCSI_PASS_THROUGH, SCSI_PASS_THROUGH::DataTransferLength)).ToString() + L" DataTransferLength");
    sb->AppendLine((offsetof(SCSI_PASS_THROUGH, SCSI_PASS_THROUGH::TimeOutValue)).ToString() + L" TimeOutValue");
    sb->AppendLine((offsetof(SCSI_PASS_THROUGH, SCSI_PASS_THROUGH::DataBufferOffset)).ToString() + L" DataBufferOffset");
    sb->AppendLine((offsetof(SCSI_PASS_THROUGH, SCSI_PASS_THROUGH::SenseInfoOffset)).ToString() + L" SenseInfoOffset");
    sb->AppendLine((offsetof(SCSI_PASS_THROUGH, SCSI_PASS_THROUGH::Cdb[0])).ToString() + L" Cdb[0]");
    sb->AppendLine(L"**TOTAL SIZE: " + (sizeof(SCSI_PASS_THROUGH)).ToString());
    sb->AppendLine();
    sb->AppendLine();
    Console::WriteLine(sb->ToString());
    Clipboard::SetText(sb->ToString());
    Console::ReadKey();
    return 0;
}

6) It should build and run. It sets the clipboard with the text. You can’t dump out the offset of bit fields, like you get in the CDROM_READ_TOC_EX structure. You should dump out x86 and x64 versions so that you can check your .Net structures match on both.

Let’s say you’ve made your .Net structure, called ScsiPassThrough, and want to see the offset of the Length field and the size. You would use the SizeOf and OffsetOf methods of the Marshal class which is in the System.Runtime.InteropServices namespace. e.g.:

Console.WriteLine(Marshal.OffsetOf(GetType(ScsiPassThrough), "Length").ToString)
Console.WriteLine(Marshal.SizeOf(GetType(ScsiPassThrough)))

Ok, back to the task in hand. We want to send IOCTL_CDROM_READ_TOC_EX which has the value &H24054. The documentation says we have to send a structure CDROM_READ_TOC_EX to specify what exactly we want to get back from the TOC.

typedef struct _CDROM_READ_TOC_EX {
    UCHAR Format    : 4;
    UCHAR Reserved1 : 3; // future expansion
    UCHAR Msf       : 1;
    UCHAR SessionTrack;
    UCHAR Reserved2;     // future expansion
    UCHAR Reserved3;     // future expansion
} CDROM_READ_TOC_EX, *PCDROM_READ_TOC_EX;

This structure has a size of 4 bytes on x86 and x64. We need to set the Format field to CDROM_READ_TOC_EX_FORMAT_CDTEXT (value 5) to say we want to retrieve CD Text. All the others will be 0. The structure is made up of 4 bytes. SessionTrack, Reserved2 and Reserved3 get a byte each. The other byte is divided up into 3 fields – Format, Reserved1 and Msf. Format takes the lo 4 bits of the byte. Reserved takes the next 3 and Msf takes the hi bit. We can’t split fields up in .Net like that. We can either declare a structure that has 4 byte fields, and expose the three bit fields with properties, or, as we just want to set the one field, we could also just send an integer with the correct bit pattern - an integer has the same size and all the dll that we are calling cares about is that it gets 4 bytes with the correct bits set. I came up with this:

<StructLayout(LayoutKind.Sequential)>
Friend Structure CDROM_READ_TOC_EX
    Public FormatReservedMsf As Byte
    Public SessionTrack As Byte
    Public Reserved2 As Byte
    Public Reserved3 As Byte
 
    ' properties will just read the public fields without contributing to the size of the structure.
    Public ReadOnly Property Format As ReadTocExFormat
        Get
            Return CType(FormatReservedMsf And &HF, ReadTocExFormat)
        End Get
    End Property
 
    Public ReadOnly Property Reserved1 As Integer
        Get
            Return (FormatReservedMsf >> 4) And &H7
        End Get
    End Property
 
    Public ReadOnly Property Msf As Boolean
        Get
            Return (FormatReservedMsf And &H80) = &H80
        End Get
    End Property
 
    Public Sub SetFormatReservedMsf(format As ReadTocExFormat, msf As Boolean)
        FormatReservedMsf = CByte(format Or If(msf, &H80, 0))
    End Sub
 
    Public ReadOnly Property Size As Integer
        Get
            Return 4
        End Get
    End Property
 
End Structure

The documentation says that when we send the message we should also send a buffer. On return the buffer will contain a CDROM_TOC_CD_TEXT_DATA structure, followed by an array of CDROM_TOC_CD_TEXT_DATA_BLOCK structures. So – how big should the buffer be to hold all this? The answer is that we don’t know. We can either create a huge buffer that will be big enough, or we can call once to read the CDROM_TOC_CD_TEXT_DATA.Length field and use that to size the buffer. The buffer can be an array of bytes – they are easy to marshal if one of the parameters states the size of the array. DeviceIOControl will look like this:

<DllImport("kernel32", SetLastError:=True)>
Public Shared Function DeviceIoControl(hVol As SafeFileHandle, controlCode As Integer,
   ByRef inbuffer As CDROM_READ_TOC_EX, inBufferSize As Integer,
   <MarshalAs(UnmanagedType.LPArray, SizeParamIndex:=6)> ByVal outBuffer() As Byte,
   outBufferSize As Integer,
   <Out()> ByRef bytesReturned As Integer, ovelapped As IntPtr) As Boolean
End Function

Note the structure goes ByRef – it’s a value type and we don’t want to send the whole structure to the function as it is expecting a pointer to the structure. Passing it ByRef tells the marshaller to send a pointer. The array, being a reference type, goes ByValue.

So the first time we call it we want to just get the Length field which is a WORD field, 2 bytes. Then we can call it again with a byte array that is this Length field + 2 as the value of the Length field is the size not including itself. The bytes are the wrong way round to read as a UShort with the BitConverter. I guess you could reverse the array. At this point my program gets the bytes and doesn’t do anything with them…

Imports System.IO
Imports System.Runtime.InteropServices
 
Public Class CdTextRetriever
 
    Private Const IOCTL_CDROM_READ_TOC_EX As UInteger = &H24054
 
    Public Shared Function GetCdText(driveInfo As DriveInfo) As Object
        Dim cdText As Object = Nothing ' don't know what the return will look like yet.
        Dim devicePath As String = String.Format("\\.\{0}:", driveInfo.Name(0))
        Const FILE_ATTRIBUTE_NORMAL As UInteger = &H80
        Const GENERIC_READ As UInteger = &H80000000UI
        Const GENERIC_WRITE As UInteger = &H40000000
        Const FILE_SHARE_READ As UInteger = 1
        Const FILE_SHARE_WRITE As UInteger = 2
        Const OPEN_EXISTING As UInteger = 3
        Using hDevice As SafeFileHandle = NativeMethods.CreateFile(devicePath,
                GENERIC_READ Or GENERIC_WRITE,
                FILE_SHARE_READ Or FILE_SHARE_WRITE,
                IntPtr.Zero, OPEN_EXISTING,
                FILE_ATTRIBUTE_NORMAL, IntPtr.Zero)
            If hDevice.IsInvalid OrElse hDevice.IsClosed Then Throw New Win32Exception
            ReadCdText(hDevice)
        End Using
        Return cdText
    End Function
 
 
    Private Shared Sub ReadCdText(hDevice As SafeFileHandle)
        Dim size As Integer = GetCdTextSize(hDevice)
        Dim bytes(size - 1) As Byte
        ReadCdText(hDevice, bytes)
        Debugger.Break()
        ' TODO: Decode the bytes!
    End Sub
 
    Private Shared Function GetCdTextSize(hDevice As SafeFileHandle) As Integer
        Dim bytes(1) As Byte ' 2 bytes to read the size of the buffer required
        ReadCdText(hDevice, bytes)
        Return (CType(bytes(0), Integer) << 8) Or bytes(1) + 2
    End Function
 
    Private Shared Sub ReadCdText(hDevice As SafeFileHandle, bytes() As Byte)
        Dim request As New CDROM_READ_TOC_EX
        request.SetFormatReservedMsf(ReadTocExFormat.CDROM_READ_TOC_EX_FORMAT_CDTEXT, False)
        Dim returned As Integer = 0
        Dim result As Boolean = DeviceIoControl(hDevice, IOCTL_CDROM_READ_TOC_EX, request,
                                                 request.Size, bytes, bytes.Length,
                                                 returned, IntPtr.Zero)
        If result = False Then Throw New Win32Exception
    End Sub
 
End Class

12 February 2011

Sending the SCSI INQUIRY command with DeviceIOControl

Option Strict On
Option Explicit On

Imports System.Runtime.InteropServices
Imports Microsoft.Win32.SafeHandles
Imports System.Security
Imports System.ComponentModel
Imports System.Text

Module Module1

    <SuppressUnmanagedCodeSecurity()> _
    Private Class NativeMethods
        <DllImport("kernel32", SetLastError:=True)> _
        Public Shared Function CreateFile( _
         ByVal FileName As String, _
         ByVal DesiredAccess As Integer, _
         ByVal ShareMode As Integer, _
         ByVal SecurityAttributes As IntPtr, _
         ByVal CreationDisposition As Integer, _
         ByVal FlagsAndAttributes As Integer, _
         ByVal hTemplateFile As IntPtr) As SafeFileHandle
        End Function

        <DllImport("kernel32.dll", SetLastError:=True)> _
        Friend Shared Function DeviceIoControl( _
            ByVal deviceHandle As SafeFileHandle, _
            ByVal controlCode As Integer, _
            ByRef inBuffer As ScsiPassThroughWithBuffers, _
            ByVal inBufferSize As Integer, _
            ByRef outBuffer As ScsiPassThroughWithBuffers, _
            ByVal outBufferSize As Integer, _
            ByRef bytesReturned As Integer, _
            ByVal overlapped1 As IntPtr) As Boolean
        End Function
    End Class

    <StructLayout(LayoutKind.Sequential, pack:=8)> _
    Private Structure ScsiPassThrough
        Public Length As Short
        Public ScsiStatus As Byte
        Public PathId As Byte          ' port / bus
        Public TargetId As Byte        ' controller
        Public Lun As Byte             ' Lun
        Public CdbLength As Byte
        Public SenseInfoLength As Byte
        Public DataIn As Byte
        Public DataTransferLength As Integer
        Public TimeOutValue As Integer
        Public DataBufferOffset As IntPtr
        Public SenseInfoOffset As Integer
        <MarshalAs(UnmanagedType.ByValArray, SizeConst:=16)> _
        Public Cdb() As Byte
        Public Sub Init()
            Me.Length = CShort(Marshal.SizeOf(GetType(ScsiPassThrough)))
            CdbLength = 6                   ' Size of our command structure
            SenseInfoLength = 32            ' Size of our buffer that may be filled with error/status info
            DataIn = 1                      ' SCSI_IOCTL_DATA_IN 
            DataTransferLength = 128        ' Size of our buffer that gets filled with informaion
            TimeOutValue = 10
            ' TargetId, PathId and Lun are filled on return. Don't mean anything on the way out.
            DataBufferOffset = Marshal.OffsetOf(GetType(ScsiPassThroughWithBuffers), "Data")
            SenseInfoOffset = Marshal.OffsetOf(GetType(ScsiPassThroughWithBuffers), "Sense").ToInt32
            Cdb = New Byte(15) {}
            Cdb(0) = &H12
            Cdb(4) = 128
        End Sub
    End Structure

    <StructLayout(LayoutKind.Sequential, pack:=8)> _
    Private Structure ScsiPassThroughWithBuffers
        Public Spt As ScsiPassThrough
        <MarshalAs(UnmanagedType.ByValArray, SizeConst:=32)> _
        Public Sense() As Byte
        <MarshalAs(UnmanagedType.ByValArray, SizeConst:=128)> _
        Public Data() As Byte
        Public Sub Init()
            Sense = New Byte(31) {}
            Data = New Byte(127) {}
            Spt = New ScsiPassThrough
            Spt.Init()
        End Sub
    End Structure

    Private Enum PeripheralDeviceType
        Sbc2 = 0 ' direct access block device (disk)
        Ssc2 = 1 ' sequential access block device (tape)
        Ssc = 2  ' Printer
        Spc2 = 3 ' Processor
        Sbc = 4  ' Write once
        Mmc = 5  ' CD/DVD etc
        SbcOptical = 7 ' Optical memory device
        Smc2 = 8 ' Medium changer
        scc2 = &HC ' storage array controller (raid)
        Ses = &HD  ' enclosure services device
        Rbc = &HE  ' simple direct access device
        Ocrw = &HF ' optical card reader
        Bcc = &H10 ' bridge controller 
        Osd = &H11 ' object based storage device
        Adc = &H12 ' automation / drive interface
        WellKnown = &H1E
        Unknown = &H1F
    End Enum

    Sub Main()
        Console.WriteLine("Enter a drive letter")
        Dim letter As Char = Console.ReadKey.KeyChar
        Console.WriteLine()
        Const GenericRead As Integer = &H80000000
        Const GenericWrite As Integer = &H40000000
        Const FileShareRead As Integer = 1
        Const FileShareWrite As Integer = 2
        Const OpenExisting As Integer = 3
        Dim drivePath As String = String.Concat("\\.\" & letter & ":")
        Console.WriteLine("Trying path: " & drivePath)
        Using driveHandle As SafeFileHandle = NativeMethods.CreateFile( _
         drivePath, _
         GenericRead Or GenericWrite, _
         FileShareRead Or FileShareWrite, _
         IntPtr.Zero, _
         OpenExisting, _
         0, _
         IntPtr.Zero)
            If driveHandle.IsInvalid Then
                Console.WriteLine("CreateFile ERROR: " & (New Win32Exception).Message)
                Console.ReadKey()
                Return
            End If
            Dim sptwb As New ScsiPassThroughWithBuffers
            sptwb.Init()

            Dim inBufferSize As Integer = Marshal.SizeOf(GetType(ScsiPassThroughWithBuffers))
            Dim bytesReturned As Integer
            Const IOCTL_SCSI_PASS_THROUGH As Integer = &H4D004
            Dim result As Boolean = NativeMethods.DeviceIoControl(driveHandle, IOCTL_SCSI_PASS_THROUGH, _
                sptwb, inBufferSize, sptwb, inBufferSize, bytesReturned, IntPtr.Zero)
            If result = False Then
                Console.WriteLine("DeviceIOControl ERROR: {0} {1}", Marshal.GetLastWin32Error.ToString("x"), (New Win32Exception).Message)
                Console.ReadKey()
                Return
            End If
            Dim bytes() As Byte = sptwb.Data
            Dim peripheralQualifier As Integer = bytes(0) >> 5
            If peripheralQualifier = 1 Then Console.WriteLine("No peripheral is attached to the device")
            Dim peripheralDevice As PeripheralDeviceType = CType(bytes(0) And &H1F, PeripheralDeviceType)
            Console.WriteLine("Peripheral device type: " & peripheralDevice.ToString)
            Console.WriteLine("Version of the standard:" & sptwb.Data(2))
            Console.WriteLine("Additional Length: " & sptwb.Data(4))
            DumpString("Vendor Id: ", sptwb.Data, 8, 8)
            DumpString("Product Id: ", sptwb.Data, 16, 16)
            DumpString("Product Revision Level: ", sptwb.Data, 32, 4)
            DumpString("Serial no: ", sptwb.Data, 36, 8)
            Console.WriteLine("Press any key")
            Console.ReadKey()
        End Using
    End Sub

    Private Sub DumpString(msg As String, bytes() As Byte, offset As Integer, length As Integer)
        Console.WriteLine(String.Format("{0} :'{1}'", msg, ASCIIEncoding.ASCII.GetString(bytes, offset, length)))
    End Sub

End Module

The serial number is unreliable. The peripheral device type tells you which standard the device is using. I’m trying to work out how to get the serial no of my raided or USB drives. This command is getting back the serial number of my controller card for my raid array when I ask it for \\.\C:

Enter a drive letter
c
Trying path: \\.\c:
Peripheral device type: Sbc2
Version of the standard:4
Additional Length: 43
Vendor Id:  :'AMD     '
Product Id:  :'2+0 Stripe/RAID0'
Product Revision Level:  :'1.10'
Serial no:  :'98031612'
Press any key

Getting Hard disk drive info with DeviceIOControl

It’s a pretty popular question on forums – how do I get the serial number for a hard disk drive. The serial number is then used for some form of homebrew security. The consensus amongst those who’ve been programming a while is that it is a waste of time and effort, unreliable and not particularly good at securing your programs…

Anyway, as a fun challenge I had a go at getting the serial number from my laptops ATA-attached drive, by using DeviceIOControl to send the IDENTIFY DEVICE ATA command, as defined in the ATA spec. This won’t work on SCSI drives, or raid or …,… ,…, so it’s not reliable and isn’t the answer for those of you looking for a unique id (there isn’t a reliable unique id).

Imports System.Runtime.InteropServices
Imports Microsoft.Win32.SafeHandles
Imports System.Security
Imports System.ComponentModel
Imports System.Text
 
Module Module1
 
    <SuppressUnmanagedCodeSecurity()> _
    Private Class NativeMethods
        <DllImport("kernel32", SetLastError:=True)> _
        Public Shared Function CreateFile( _
         ByVal FileName As String, _
         ByVal DesiredAccess As Integer, _
         ByVal ShareMode As Integer, _
         ByVal SecurityAttributes As IntPtr, _
         ByVal CreationDisposition As Integer, _
         ByVal FlagsAndAttributes As Integer, _
         ByVal hTemplateFile As IntPtr) As SafeFileHandle
        End Function
 
        <DllImport("kernel32.dll", SetLastError:=True)> _
        Friend Shared Function DeviceIoControl( _
            ByVal deviceHandle As SafeFileHandle, _
            ByVal controlCode As Integer, _
            ByRef inBuffer As ATA_PASS_THROUGH_EX_WITH_BUFFERS, _
            ByVal inBufferSize As Integer, _
            ByRef outBuffer As ATA_PASS_THROUGH_EX_WITH_BUFFERS, _
            ByVal outBufferSize As Integer, _
            ByRef bytesReturned As Integer, _
            ByVal overlapped1 As IntPtr) As Boolean
        End Function
    End Class
 
    <StructLayout(LayoutKind.Sequential)> _
    Private Structure ATA_PASS_THROUGH_EX
        Public Length As Short
        Public AtaFlags As Short
        Public PathId As Byte
        Public TargetId As Byte
        Public Lun As Byte
        Public ReservedAsUchar As Byte
        Public DataTransferLength As Integer
        Public TimeOutValue As Integer
        Public ReservedAsUlong As Integer
        Public DataBufferOffset As IntPtr
        <MarshalAs(UnmanagedType.ByValArray, sizeconst:=8)> _
        Public PreviousTaskFile() As Byte
        <MarshalAs(UnmanagedType.ByValArray, sizeconst:=8)> _
        Public CurrentTaskFile() As Byte
    End Structure
 
    <StructLayout(LayoutKind.Sequential)> _
    Private Structure ATA_PASS_THROUGH_EX_WITH_BUFFERS
        Public Apt As ATA_PASS_THROUGH_EX
        <MarshalAs(UnmanagedType.ByValArray, sizeconst:=512)> _
        Public Data() As Byte
    End Structure
 
    Sub Main()
        Console.WriteLine("Enter a drive letter")
        Dim letter As Char = Console.ReadKey.KeyChar
        Console.WriteLine()
        Const GenericRead As Integer = &H80000000
        Const GenericWrite As Integer = &H40000000
        Const FileShareRead As Integer = 1
        Const FileShareWrite As Integer = 2
        Const OpenExisting As Integer = 3
        Dim drivePath As String = String.Concat("\\.\" & letter & ":")
        Console.WriteLine("Trying path: " & drivePath)
        Using driveHandle As SafeFileHandle = NativeMethods.CreateFile( _
         drivePath, _
         GenericRead Or GenericWrite, _
         FileShareRead Or FileShareWrite, _
         IntPtr.Zero, _
         OpenExisting, _
         0, _
         IntPtr.Zero)
            If driveHandle.IsInvalid Then
                Console.WriteLine("CreateFile ERROR: " & (New Win32Exception).Message)
                Console.ReadKey()
                Return
            End If
            Dim apex As New ATA_PASS_THROUGH_EX
            apex.Length = Marshal.SizeOf(apex)
            apex.AtaFlags = 2 ' ATA_FLAGS_DATA_IN
            apex.DataTransferLength = 512 ' The command returns a 512 byte package of info.
            apex.TimeOutValue = 10 ' 10 second timeout.
            apex.DataBufferOffset = Marshal.OffsetOf(GetType(ATA_PASS_THROUGH_EX_WITH_BUFFERS), "Data")
            apex.CurrentTaskFile = New Byte(7) {} ' This contains the command we are requesting.
            apex.CurrentTaskFile(6) = &HEC        ' <-- the command "IDENTIFY DEVICE"
            Dim apexb As New ATA_PASS_THROUGH_EX_WITH_BUFFERS
            apexb.Apt = apex
            Dim inBufferSize As Integer = Marshal.SizeOf(GetType(ATA_PASS_THROUGH_EX_WITH_BUFFERS))
            Dim bytesReturned As Integer
            Const IOCTL_ATA_PASS_THROUGH As Integer = &H4D02C
            Dim result As Boolean = NativeMethods.DeviceIoControl(driveHandle, IOCTL_ATA_PASS_THROUGH, _
                apexb, inBufferSize, apexb, inBufferSize, bytesReturned, IntPtr.Zero)
            If result = False Then
                Console.WriteLine("DeviceIOControl ERROR: " & (New Win32Exception).Message)
                Console.ReadKey()
                Return
            End If
            DumpString("S/N: ", apexb.Data, 20, 20)
            DumpString("Firmware: ", apexb.Data, 46, 8)
            DumpString("Model: ", apexb.Data, 54, 40)
            Console.WriteLine("Press any key")
            Console.ReadKey()
        End Using
    End Sub
 
    Private Sub DumpString(msg As String, bytes() As Byte, offset As Integer, length As Integer)
        ' The strings are slightly weird - endianness? If you use ASCII.GetBytes then each character 
        ' pair is reversed.
        Dim sb As New StringBuilder(msg & " '")
        For i As Integer = offset To offset + length - 1 Step 2
            sb.Append(Chr(bytes(i + 1)))
            sb.Append(Chr(bytes(i)))
        Next
        sb.Append("'"c)
        Console.WriteLine(sb.ToString)
    End Sub
 
End Module

Output on my laptop:

Enter a drive letter
c
Trying path: \\.\c:
S/N:  '            5TG077W9'
Firmware:  'DE14    '
Model:  'ST9160411ASG                            '
Press any key