Jahidul Arafat's Blog
Node.js-Blog#04 Buffers: What is uninitialized buffer and which buffer class method basically return it?

Node.js-Blog#04 Buffers: What is uninitialized buffer and which buffer class method basically return it?

Jahidul Arafat's photo
Jahidul Arafat
·

8 min read

Suppose, suddenly you have been asked what is uninitialized buffer and which buffer class method basically return it? Answer of this question could be very simple, but requires a good understanding of Node'js Buffer. That's what my this blog is all about.

In this blog, following topics will be covered:

  • Why Node.js Buffer?
  • What are Buffers?
  • Where you see Buffers?
  • Operations with Buffer: {Create Buffer, Initializing it, Writing to it, reading from it}
  • More funs with Buffer

1. Why Node.js Buffers?

  • Pure JavaScript, while great with 'unicode-encoded strings', does not handle straight 'binary data' very well.
  • This is fine on the browser, where most data is in the form of strings.
  • However, Node.js servers have to also deal with TCP streams and reading and writing to the filesystem, both of which make it necessary to deal with purely binary streams of data.

    A Problematic Approach: One way to handle this problem is to just use strings anyway, which is exactly what Node.js did at first

  • However, this approach is extremely problematic to work with; It's slow, makes you work with an API designed for strings and not binary data, and has a tendency to break in strange and mysterious ways.

    Don't use binary strings. Use buffers instead!

2. What Are Buffers?

  • The Buffer class in Node.js is designed to handle 'raw binary data'. Each buffer corresponds to some raw memory allocated outside V8.
  • Buffers act somewhat like arrays of integers, but aren't resizable and have a whole bunch of methods specifically for binary data.
  • The integers in a buffer each represent a byte and so are limited to values from 0 to 255 inclusive.

    When using console.log() to print the Buffer instance, you'll get a chain of values in hexadecimal values.

2.1 Where You See Buffers: My simple example

  • In the wild, buffers are usually seen in the context of binary data coming from streams, such as fs.createReadStream.

3. Usage of Buffer

3.1 Creating a Buffer

//A. Create a new buffer of 8 bytes of zeros
var buffer = Buffer.alloc(8);
// Output: This will print out 8 bytes of zero:
// <Buffer 00 00 00 00 00 00 00 00>

//B. Create a buffer with 8 bytes of certain value
var buffer = Buffer.from([ 8, 6, 7, 5, 3, 0, 9]);
// This will print out 8 bytes of certain values:
// <Buffer 08 06 07 05 03 00 09>

//C.  initializes the buffer to a binary encoding of the first string as specified by the second argument
var buffer = Buffer.from("I'm a string!", "utf-8");
// There are 13 chars including space
// This will print out a chain of values in utf-8:
// <Buffer 49 27 6d 20 61 20 73 74 72 69 6e 67 21>
// 'utf-8' is by far the most common encoding used with Node.js, but Buffer also supports others.

3.2 Writing to Buffers

//A. Given that there is already a buffer created with 16 bytes of Zeros
var buffer = Buffer.alloc(16)

//B. write a string to it
buffer.write("Hello", "utf-8") //(string,encoding)
// it returns 5, means we wrote to five bytes of the buffer.

//C. When buffer.write has 3 arguments, the second argument indicates an offset, or the index of the buffer to start writing at.
buffer.write(" world!", 5, "utf-8") //(string,offset,encoding)
// it returns 7, means we wrote to seven bytes of the buffer starting from 5th position.

3.3 Reading from Buffers

//A. Probably the most common way to read buffers is to use the toString method, since many buffers contain text:
buffer.toString('utf-8')
// returns 'Hello world!\u0000�k\t', see not the entire buffer is used and empty become gurbage

//B. Luckily, because we know how many bytes we've written to the buffer, we can simply add more arguments to "stringify" the slice that's actually interesting:
buffer.toString("utf-8", 0, 12)
// returns 'Hello world!'

//C. You can also set individual bytes by using an array-like syntax:
buffer[15] = 33

4 More fun with Buffers

  • Buffer.isBuffer(object) - This method checks to see if object is a buffer, similar to Array.isArray.
  • Buffer.byteLength(string, encoding) - check the number of bytes required to encode a string with a given encoding (which defaults to utf-8)

    This length is not the same as string length, since many characters require more bytes to encode.

 > var snowman = "☃";
 > snowman.length
 1
 > Buffer.byteLength(snowman)
 3

*** The unicode snowman is only one character, but takes 3 entire bytes to encode!
  • buffer.length - This is the length of your buffer, and represents how much memory is allocated.

    It is not the same as the size of the buffer's contents, since a buffer may be half-filled. For example:

> var buffer = Buffer.alloc(16)
> buffer.write(snowman)
3
> buffer.length
16

 *** In this example, the contents written to the buffer only consist of three groups (since they represent the single-character snowman), but the buffer's length is still 16, as it was initialized.
  • buffer.copy
  • buffer.slice(start, end=buffer.length)
  • Buffer.allocUnsafe(n) - The value of an unsafe buffer is not emptied, and can contain data from older buffers

    This gives the answer to our blog starting question

Example 01

var buf = Buffer.allocUnsafe(15);
/*The value of an unsafe buffer is **not emptied, and can contain data from older buffers*/
console.log(buf);

//Result: <Buffer 02 00 00 00 a7 01 00 00 10 53 49 90 02 a7 01>

Example 02

var buf = Buffer.allocUnsafe(15); /*The value of an unsafe buffer is not emptied, and can contain data from older buffers:*/
console.log(buf); // <Buffer 02 00 00 00 a7 01 00 00 10 53 49 90 02 a7 01>
/*Empty the buffer:*/
buf.fill(0);        // <Buffer 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00>
/*The buffer is now zero-filled:*/
console.log(buf);   // <Buffer 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00>
Node.js