1 # safe-buffer [![travis][travis-image]][travis-url] [![npm][npm-image]][npm-url] [![downloads][downloads-image]][downloads-url] [![javascript style guide][standard-image]][standard-url]
3 [travis-image]: https://img.shields.io/travis/feross/safe-buffer/master.svg
4 [travis-url]: https://travis-ci.org/feross/safe-buffer
5 [npm-image]: https://img.shields.io/npm/v/safe-buffer.svg
6 [npm-url]: https://npmjs.org/package/safe-buffer
7 [downloads-image]: https://img.shields.io/npm/dm/safe-buffer.svg
8 [downloads-url]: https://npmjs.org/package/safe-buffer
9 [standard-image]: https://img.shields.io/badge/code_style-standard-brightgreen.svg
10 [standard-url]: https://standardjs.com
12 #### Safer Node.js Buffer API
14 **Use the new Node.js Buffer APIs (`Buffer.from`, `Buffer.alloc`,
15 `Buffer.allocUnsafe`, `Buffer.allocUnsafeSlow`) in all versions of Node.js.**
17 **Uses the built-in implementation when available.**
22 npm install safe-buffer
27 The goal of this package is to provide a safe replacement for the node.js `Buffer`.
29 It's a drop-in replacement for `Buffer`. You can use it by adding one `require` line to
30 the top of your node.js modules:
33 var Buffer = require('safe-buffer').Buffer
35 // Existing buffer code will continue to work without issues:
37 new Buffer('hey', 'utf8')
38 new Buffer([1, 2, 3], 'utf8')
40 new Buffer(16) // create an uninitialized buffer (potentially unsafe)
42 // But you can use these new explicit APIs to make clear what you want:
44 Buffer.from('hey', 'utf8') // convert from many types to a Buffer
45 Buffer.alloc(16) // create a zero-filled buffer (safe)
46 Buffer.allocUnsafe(16) // create an uninitialized buffer (potentially unsafe)
51 ### Class Method: Buffer.from(array)
58 Allocates a new `Buffer` using an `array` of octets.
61 const buf = Buffer.from([0x62,0x75,0x66,0x66,0x65,0x72]);
62 // creates a new Buffer containing ASCII bytes
63 // ['b','u','f','f','e','r']
66 A `TypeError` will be thrown if `array` is not an `Array`.
68 ### Class Method: Buffer.from(arrayBuffer[, byteOffset[, length]])
73 * `arrayBuffer` {ArrayBuffer} The `.buffer` property of a `TypedArray` or
75 * `byteOffset` {Number} Default: `0`
76 * `length` {Number} Default: `arrayBuffer.length - byteOffset`
78 When passed a reference to the `.buffer` property of a `TypedArray` instance,
79 the newly created `Buffer` will share the same allocated memory as the
83 const arr = new Uint16Array(2);
87 const buf = Buffer.from(arr.buffer); // shares the memory with arr;
90 // Prints: <Buffer 88 13 a0 0f>
92 // changing the TypedArray changes the Buffer also
96 // Prints: <Buffer 88 13 70 17>
99 The optional `byteOffset` and `length` arguments specify a memory range within
100 the `arrayBuffer` that will be shared by the `Buffer`.
103 const ab = new ArrayBuffer(10);
104 const buf = Buffer.from(ab, 0, 2);
105 console.log(buf.length);
109 A `TypeError` will be thrown if `arrayBuffer` is not an `ArrayBuffer`.
111 ### Class Method: Buffer.from(buffer)
118 Copies the passed `buffer` data onto a new `Buffer` instance.
121 const buf1 = Buffer.from('buffer');
122 const buf2 = Buffer.from(buf1);
125 console.log(buf1.toString());
127 console.log(buf2.toString());
128 // 'buffer' (copy is not changed)
131 A `TypeError` will be thrown if `buffer` is not a `Buffer`.
133 ### Class Method: Buffer.from(str[, encoding])
138 * `str` {String} String to encode.
139 * `encoding` {String} Encoding to use, Default: `'utf8'`
141 Creates a new `Buffer` containing the given JavaScript string `str`. If
142 provided, the `encoding` parameter identifies the character encoding.
143 If not provided, `encoding` defaults to `'utf8'`.
146 const buf1 = Buffer.from('this is a tést');
147 console.log(buf1.toString());
148 // prints: this is a tést
149 console.log(buf1.toString('ascii'));
150 // prints: this is a tC)st
152 const buf2 = Buffer.from('7468697320697320612074c3a97374', 'hex');
153 console.log(buf2.toString());
154 // prints: this is a tést
157 A `TypeError` will be thrown if `str` is not a string.
159 ### Class Method: Buffer.alloc(size[, fill[, encoding]])
165 * `fill` {Value} Default: `undefined`
166 * `encoding` {String} Default: `utf8`
168 Allocates a new `Buffer` of `size` bytes. If `fill` is `undefined`, the
169 `Buffer` will be *zero-filled*.
172 const buf = Buffer.alloc(5);
174 // <Buffer 00 00 00 00 00>
177 The `size` must be less than or equal to the value of
178 `require('buffer').kMaxLength` (on 64-bit architectures, `kMaxLength` is
179 `(2^31)-1`). Otherwise, a [`RangeError`][] is thrown. A zero-length Buffer will
180 be created if a `size` less than or equal to 0 is specified.
182 If `fill` is specified, the allocated `Buffer` will be initialized by calling
183 `buf.fill(fill)`. See [`buf.fill()`][] for more information.
186 const buf = Buffer.alloc(5, 'a');
188 // <Buffer 61 61 61 61 61>
191 If both `fill` and `encoding` are specified, the allocated `Buffer` will be
192 initialized by calling `buf.fill(fill, encoding)`. For example:
195 const buf = Buffer.alloc(11, 'aGVsbG8gd29ybGQ=', 'base64');
197 // <Buffer 68 65 6c 6c 6f 20 77 6f 72 6c 64>
200 Calling `Buffer.alloc(size)` can be significantly slower than the alternative
201 `Buffer.allocUnsafe(size)` but ensures that the newly created `Buffer` instance
202 contents will *never contain sensitive data*.
204 A `TypeError` will be thrown if `size` is not a number.
206 ### Class Method: Buffer.allocUnsafe(size)
213 Allocates a new *non-zero-filled* `Buffer` of `size` bytes. The `size` must
214 be less than or equal to the value of `require('buffer').kMaxLength` (on 64-bit
215 architectures, `kMaxLength` is `(2^31)-1`). Otherwise, a [`RangeError`][] is
216 thrown. A zero-length Buffer will be created if a `size` less than or equal to
219 The underlying memory for `Buffer` instances created in this way is *not
220 initialized*. The contents of the newly created `Buffer` are unknown and
221 *may contain sensitive data*. Use [`buf.fill(0)`][] to initialize such
222 `Buffer` instances to zeroes.
225 const buf = Buffer.allocUnsafe(5);
227 // <Buffer 78 e0 82 02 01>
228 // (octets will be different, every time)
231 // <Buffer 00 00 00 00 00>
234 A `TypeError` will be thrown if `size` is not a number.
236 Note that the `Buffer` module pre-allocates an internal `Buffer` instance of
237 size `Buffer.poolSize` that is used as a pool for the fast allocation of new
238 `Buffer` instances created using `Buffer.allocUnsafe(size)` (and the deprecated
239 `new Buffer(size)` constructor) only when `size` is less than or equal to
240 `Buffer.poolSize >> 1` (floor of `Buffer.poolSize` divided by two). The default
241 value of `Buffer.poolSize` is `8192` but can be modified.
243 Use of this pre-allocated internal memory pool is a key difference between
244 calling `Buffer.alloc(size, fill)` vs. `Buffer.allocUnsafe(size).fill(fill)`.
245 Specifically, `Buffer.alloc(size, fill)` will *never* use the internal Buffer
246 pool, while `Buffer.allocUnsafe(size).fill(fill)` *will* use the internal
247 Buffer pool if `size` is less than or equal to half `Buffer.poolSize`. The
248 difference is subtle but can be important when an application requires the
249 additional performance that `Buffer.allocUnsafe(size)` provides.
251 ### Class Method: Buffer.allocUnsafeSlow(size)
258 Allocates a new *non-zero-filled* and non-pooled `Buffer` of `size` bytes. The
259 `size` must be less than or equal to the value of
260 `require('buffer').kMaxLength` (on 64-bit architectures, `kMaxLength` is
261 `(2^31)-1`). Otherwise, a [`RangeError`][] is thrown. A zero-length Buffer will
262 be created if a `size` less than or equal to 0 is specified.
264 The underlying memory for `Buffer` instances created in this way is *not
265 initialized*. The contents of the newly created `Buffer` are unknown and
266 *may contain sensitive data*. Use [`buf.fill(0)`][] to initialize such
267 `Buffer` instances to zeroes.
269 When using `Buffer.allocUnsafe()` to allocate new `Buffer` instances,
270 allocations under 4KB are, by default, sliced from a single pre-allocated
271 `Buffer`. This allows applications to avoid the garbage collection overhead of
272 creating many individually allocated Buffers. This approach improves both
273 performance and memory usage by eliminating the need to track and cleanup as
274 many `Persistent` objects.
276 However, in the case where a developer may need to retain a small chunk of
277 memory from a pool for an indeterminate amount of time, it may be appropriate
278 to create an un-pooled Buffer instance using `Buffer.allocUnsafeSlow()` then
279 copy out the relevant bits.
282 // need to keep around a few small chunks of memory
285 socket.on('readable', () => {
286 const data = socket.read();
287 // allocate for retained data
288 const sb = Buffer.allocUnsafeSlow(10);
289 // copy the data into the new allocation
290 data.copy(sb, 0, 0, 10);
295 Use of `Buffer.allocUnsafeSlow()` should be used only as a last resort *after*
296 a developer has observed undue memory retention in their applications.
298 A `TypeError` will be thrown if `size` is not a number.
302 The rest of the `Buffer` API is exactly the same as in node.js.
303 [See the docs](https://nodejs.org/api/buffer.html).
308 - [Node.js issue: Buffer(number) is unsafe](https://github.com/nodejs/node/issues/4660)
309 - [Node.js Enhancement Proposal: Buffer.from/Buffer.alloc/Buffer.zalloc/Buffer() soft-deprecate](https://github.com/nodejs/node-eps/pull/4)
311 ## Why is `Buffer` unsafe?
313 Today, the node.js `Buffer` constructor is overloaded to handle many different argument
314 types like `String`, `Array`, `Object`, `TypedArrayView` (`Uint8Array`, etc.),
315 `ArrayBuffer`, and also `Number`.
317 The API is optimized for convenience: you can throw any type at it, and it will try to do
320 Because the Buffer constructor is so powerful, you often see code like this:
323 // Convert UTF-8 strings to hex
324 function toHex (str) {
325 return new Buffer(str).toString('hex')
329 ***But what happens if `toHex` is called with a `Number` argument?***
331 ### Remote Memory Disclosure
333 If an attacker can make your program call the `Buffer` constructor with a `Number`
334 argument, then they can make it allocate uninitialized memory from the node.js process.
335 This could potentially disclose TLS private keys, user data, or database passwords.
337 When the `Buffer` constructor is passed a `Number` argument, it returns an
338 **UNINITIALIZED** block of memory of the specified `size`. When you create a `Buffer` like
339 this, you **MUST** overwrite the contents before returning it to the user.
341 From the [node.js docs](https://nodejs.org/api/buffer.html#buffer_new_buffer_size):
347 > The underlying memory for `Buffer` instances created in this way is not initialized.
348 > **The contents of a newly created `Buffer` are unknown and could contain sensitive
349 > data.** Use `buf.fill(0)` to initialize a Buffer to zeroes.
353 Whenever the programmer intended to create an uninitialized `Buffer` you often see code
357 var buf = new Buffer(16)
359 // Immediately overwrite the uninitialized buffer with data from another buffer
360 for (var i = 0; i < buf.length; i++) {
366 ### Would this ever be a problem in real code?
368 Yes. It's surprisingly common to forget to check the type of your variables in a
369 dynamically-typed language like JavaScript.
371 Usually the consequences of assuming the wrong type is that your program crashes with an
372 uncaught exception. But the failure mode for forgetting to check the type of arguments to
373 the `Buffer` constructor is more catastrophic.
375 Here's an example of a vulnerable service that takes a JSON payload and converts it to
379 // Take a JSON payload {str: "some string"} and convert it to hex
380 var server = http.createServer(function (req, res) {
382 req.setEncoding('utf8')
383 req.on('data', function (chunk) {
386 req.on('end', function () {
387 var body = JSON.parse(data)
388 res.end(new Buffer(body.str).toString('hex'))
395 In this example, an http client just has to send:
403 and it will get back 1,000 bytes of uninitialized memory from the server.
405 This is a very serious bug. It's similar in severity to the
406 [the Heartbleed bug](http://heartbleed.com/) that allowed disclosure of OpenSSL process
407 memory by remote attackers.
410 ### Which real-world packages were vulnerable?
412 #### [`bittorrent-dht`](https://www.npmjs.com/package/bittorrent-dht)
414 [Mathias Buus](https://github.com/mafintosh) and I
415 ([Feross Aboukhadijeh](http://feross.org/)) found this issue in one of our own packages,
416 [`bittorrent-dht`](https://www.npmjs.com/package/bittorrent-dht). The bug would allow
417 anyone on the internet to send a series of messages to a user of `bittorrent-dht` and get
418 them to reveal 20 bytes at a time of uninitialized memory from the node.js process.
421 [the commit](https://github.com/feross/bittorrent-dht/commit/6c7da04025d5633699800a99ec3fbadf70ad35b8)
422 that fixed it. We released a new fixed version, created a
423 [Node Security Project disclosure](https://nodesecurity.io/advisories/68), and deprecated all
424 vulnerable versions on npm so users will get a warning to upgrade to a newer version.
426 #### [`ws`](https://www.npmjs.com/package/ws)
428 That got us wondering if there were other vulnerable packages. Sure enough, within a short
429 period of time, we found the same issue in [`ws`](https://www.npmjs.com/package/ws), the
430 most popular WebSocket implementation in node.js.
432 If certain APIs were called with `Number` parameters instead of `String` or `Buffer` as
433 expected, then uninitialized server memory would be disclosed to the remote peer.
435 These were the vulnerable methods:
443 Here's a vulnerable socket server with some echo functionality:
446 server.on('connection', function (socket) {
447 socket.on('message', function (message) {
448 message = JSON.parse(message)
449 if (message.type === 'echo') {
450 socket.send(message.data) // send back the user's message
456 `socket.send(number)` called on the server, will disclose server memory.
458 Here's [the release](https://github.com/websockets/ws/releases/tag/1.0.1) where the issue
459 was fixed, with a more detailed explanation. Props to
460 [Arnout Kazemier](https://github.com/3rd-Eden) for the quick fix. Here's the
461 [Node Security Project disclosure](https://nodesecurity.io/advisories/67).
464 ### What's the solution?
466 It's important that node.js offers a fast way to get memory otherwise performance-critical
467 applications would needlessly get a lot slower.
469 But we need a better way to *signal our intent* as programmers. **When we want
470 uninitialized memory, we should request it explicitly.**
472 Sensitive functionality should not be packed into a developer-friendly API that loosely
473 accepts many different types. This type of API encourages the lazy practice of passing
474 variables in without checking the type very carefully.
476 #### A new API: `Buffer.allocUnsafe(number)`
478 The functionality of creating buffers with uninitialized memory should be part of another
479 API. We propose `Buffer.allocUnsafe(number)`. This way, it's not part of an API that
480 frequently gets user input of all sorts of different types passed into it.
483 var buf = Buffer.allocUnsafe(16) // careful, uninitialized memory!
485 // Immediately overwrite the uninitialized buffer with data from another buffer
486 for (var i = 0; i < buf.length; i++) {
492 ### How do we fix node.js core?
494 We sent [a PR to node.js core](https://github.com/nodejs/node/pull/4514) (merged as
495 `semver-major`) which defends against one case:
499 new Buffer(str, 'utf8')
502 In this situation, it's implied that the programmer intended the first argument to be a
503 string, since they passed an encoding as a second argument. Today, node.js will allocate
504 uninitialized memory in the case of `new Buffer(number, encoding)`, which is probably not
505 what the programmer intended.
507 But this is only a partial solution, since if the programmer does `new Buffer(variable)`
508 (without an `encoding` parameter) there's no way to know what they intended. If `variable`
509 is sometimes a number, then uninitialized memory will sometimes be returned.
511 ### What's the real long-term fix?
513 We could deprecate and remove `new Buffer(number)` and use `Buffer.allocUnsafe(number)` when
514 we need uninitialized memory. But that would break 1000s of packages.
516 ~~We believe the best solution is to:~~
518 ~~1. Change `new Buffer(number)` to return safe, zeroed-out memory~~
520 ~~2. Create a new API for creating uninitialized Buffers. We propose: `Buffer.allocUnsafe(number)`~~
524 We now support adding three new APIs:
526 - `Buffer.from(value)` - convert from any type to a buffer
527 - `Buffer.alloc(size)` - create a zero-filled buffer
528 - `Buffer.allocUnsafe(size)` - create an uninitialized buffer with given size
530 This solves the core problem that affected `ws` and `bittorrent-dht` which is
531 `Buffer(variable)` getting tricked into taking a number argument.
533 This way, existing code continues working and the impact on the npm ecosystem will be
534 minimal. Over time, npm maintainers can migrate performance-critical code to use
535 `Buffer.allocUnsafe(number)` instead of `new Buffer(number)`.
540 We think there's a serious design issue with the `Buffer` API as it exists today. It
541 promotes insecure software by putting high-risk functionality into a convenient API
542 with friendly "developer ergonomics".
544 This wasn't merely a theoretical exercise because we found the issue in some of the
545 most popular npm packages.
547 Fortunately, there's an easy fix that can be applied today. Use `safe-buffer` in place of
551 var Buffer = require('safe-buffer').Buffer
554 Eventually, we hope that node.js core can switch to this new, safer behavior. We believe
555 the impact on the ecosystem would be minimal since it's not a breaking change.
556 Well-maintained, popular packages would be updated to use `Buffer.alloc` quickly, while
557 older, insecure packages would magically become safe from this attack vector.
562 - [Node.js PR: buffer: throw if both length and enc are passed](https://github.com/nodejs/node/pull/4514)
563 - [Node Security Project disclosure for `ws`](https://nodesecurity.io/advisories/67)
564 - [Node Security Project disclosure for`bittorrent-dht`](https://nodesecurity.io/advisories/68)
569 The original issues in `bittorrent-dht`
570 ([disclosure](https://nodesecurity.io/advisories/68)) and
571 `ws` ([disclosure](https://nodesecurity.io/advisories/67)) were discovered by
572 [Mathias Buus](https://github.com/mafintosh) and
573 [Feross Aboukhadijeh](http://feross.org/).
575 Thanks to [Adam Baldwin](https://github.com/evilpacket) for helping disclose these issues
576 and for his work running the [Node Security Project](https://nodesecurity.io/).
578 Thanks to [John Hiesey](https://github.com/jhiesey) for proofreading this README and
584 MIT. Copyright (C) [Feross Aboukhadijeh](http://feross.org)