ruby option parser binary input

In this commodity you'll learn nearly the Ruby pack & unpack methods!

But why do nosotros need these methods?

Working with text is a lot easier than working with binary data.

Text allows you lot to apply:

regular expressions
methods like browse & lucifer
gsub

But if you want to work with binary information, there is some actress work to exercise. That's where the Array#pack & String#unpack methods come into play.

Allow me show y'all some examples, starting with just a evidently cord & and so moving on to more interesting stuff.

Cord to ASCII Values

This will convert every character in the string into a decimal value:

str = "AABBCC" str.unpack("c*")  # [65, 65, 66, 66, 67, 67]

Notice the "c*" argument for unpack.

This is a "format string" which tells unpack what to do with the information. In this instance, c ways take one graphic symbol & convert information technology into an integer value (the String#ord method also does this).

The asterisk * but says "repeat this format for all the input information".

Convert Hex to String

The H used with the pack method gives you a hex number to string conversion.

Example:

["414243"].pack("H*") # "ABC"  "ABC".unpack("H*") # ["414243"]

How to Catechumen Hex to Integer

This format cord takes 4 bytes of data & returns an integer. One affair to notice is that these bytes are in "little-endian" format.

Examples:

"\xff\x00\x00\x00".unpack("l").first # 255

"\x90\xC0\xDD\x08".unpack("fifty").starting time # 148750480

I used first here because unpack returns an array.

Binary File Parsing With The Unpack Method

How do you read a binary file like an EXE, PNG or GZIP?

If you read these files similar manifestly text, you'll see something that looks like random data…

ruby string pack

It's not random stuff.

There is a documented construction for many of these file formats & the unpack method is what yous can use to read that data and convert it into something useful.

Here is an example:

binary_data     = "\x05\x00\x68\x65\x6c\x6c\x6f" length, bulletin = binary_data.unpack("Sa*")  # [5, "hello"]

In this example, the binary information (represented in hexadecimal, which is way more compact than 1s & 0s) has a ii-byte (xvi bit) length field that contains the length of the post-obit cord. Then there is the string itself.

It is very mutual for binary files & binary network protocols to have a "length" field.

This tells the parser exactly how many bytes should be read.

Yes.

I know in this instance I read both the length & the data in one step, that's only to continue things simple.

How to Apply The BinData Gem

There is as well the bindata gem, which is congenital specifically to assistance you parse binary structures.

Here is an example:

class BinaryString < BinData::Tape   endian :piffling   uint16 :len   string :name, :read_length => :len stop

Notice the read_length parameter. This will tell bindata to work out the length from the field, and so this will save you a lot of work 🙂

And so if you lot want to write a parser for any binary format, these are the steps:

Find the specification for this format (if it's not public you volition have to reverse-engineer it, which is an entire topic on its own)
Write a `bindata` class for every section of the file (y'all will ordinarily find a header department start with metadata & then multiple data sections)
Read the data & process it yet yous want (for example, in a PNG you could change the colors of the image)
Profit!

If y'all want to encounter a full case of bindata in action take a wait at my PNG parser on github.

Base64 Encoding

At that place is this blazon of encoding called "Base64". You may accept seen it before on a URL.

Looks something like this:

U2VuZCByZWluZm9yY2VtZW50cw==

The double equals at the end is usually the tell-tale sign that you lot are dealing with Base64, although some inputs tin can result in the equals signs not being there (they are used as padding).

So why I'm telling you lot this…

Besides being a useful affair to know past itself?

Well, information technology turns out that you can convert a string into Base64 using the pack method.

Equally you lot can see hither:

def encode64(bin)   [bin].pack("m") end  encode64 "abcd"  # "YWJjZA==\n"

In fact, this is the verbal method used in the Base64 module from the standard library.

Summary

In this post, you learned most the pack & unpack methods, which help y'all piece of work with binary data. You tin can use this to parse binary files, catechumen a string into ASCII values, and for Base64 encoding.

Don't forget to share & subscribe so you can enjoy more blog post like this! 🙂