Initial Commit

This commit is contained in:
Gerrit Linnemann
2017-12-18 14:24:11 +01:00
commit 0dd4236e11
2204 changed files with 269997 additions and 0 deletions
+358
View File
@@ -0,0 +1,358 @@
# node-pre-gyp changelog
## 0.6.36
- Removed the running of `testbinary` during install. Because this was regressed for so long, it is too dangerous to re-enable by default. Developers needing validation can call `node-pre-gyp testbinary` directory.
- Fixed regression in v0.6.35 for electron installs (now skipping binary validation which is not yet supported for electron)
## 0.6.35
- No longer recommending `npm ls` in `prepublish` (#291)
- Fixed testbinary command (#283) @szdavid92
## 0.6.34
- Added new node versions to crosswalk, including v8
- Upgraded deps to latest versions, started using `^` instead of `~` for all deps.
## 0.6.33
- Improved support for yarn
## 0.6.32
- Honor npm configuration for CA bundles (@heikkipora)
- Add node-pre-gyp and npm versions to user agent (@addaleax)
- Updated various deps
- Add known node version for v7.x
## 0.6.31
- Updated various deps
## 0.6.30
- Update to npmlog@4.x and semver@5.3.x
- Add known node version for v6.5.0
## 0.6.29
- Add known node versions for v0.10.45, v0.12.14, v4.4.4, v5.11.1, and v6.1.0
## 0.6.28
- Now more verbose when remote binaries are not available. This is needed since npm is increasingly more quiet by default
and users need to know why builds are falling back to source compiles that might then error out.
## 0.6.27
- Add known node version for node v6
- Stopped bundling dependencies
- Documented method for module authors to avoid bundling node-pre-gyp
- See https://github.com/mapbox/node-pre-gyp/tree/master#configuring for details
## 0.6.26
- Skip validation for nw runtime (https://github.com/mapbox/node-pre-gyp/pull/181) via @fleg
## 0.6.25
- Improved support for auto-detection of electron runtime in `node-pre-gyp.find()`
- Pull request from @enlight - https://github.com/mapbox/node-pre-gyp/pull/187
- Add known node version for 4.4.1 and 5.9.1
## 0.6.24
- Add known node version for 5.8.0, 5.9.0, and 4.4.0.
## 0.6.23
- Add known node version for 0.10.43, 0.12.11, 4.3.2, and 5.7.1.
## 0.6.22
- Add known node version for 4.3.1, and 5.7.0.
## 0.6.21
- Add known node version for 0.10.42, 0.12.10, 4.3.0, and 5.6.0.
## 0.6.20
- Add known node version for 4.2.5, 4.2.6, 5.4.0, 5.4.1,and 5.5.0.
## 0.6.19
- Add known node version for 4.2.4
## 0.6.18
- Add new known node versions for 0.10.x, 0.12.x, 4.x, and 5.x
## 0.6.17
- Re-tagged to fix packaging problem of `Error: Cannot find module 'isarray'`
## 0.6.16
- Added known version in crosswalk for 5.1.0.
## 0.6.15
- Upgraded tar-pack (https://github.com/mapbox/node-pre-gyp/issues/182)
- Support custom binary hosting mirror (https://github.com/mapbox/node-pre-gyp/pull/170)
- Added known version in crosswalk for 4.2.2.
## 0.6.14
- Added node 5.x version
## 0.6.13
- Added more known node 4.x versions
## 0.6.12
- Added support for [Electron](http://electron.atom.io/). Just pass the `--runtime=electron` flag when building/installing. Thanks @zcbenz
## 0.6.11
- Added known node and io.js versions including more 3.x and 4.x versions
## 0.6.10
- Added known node and io.js versions including 3.x and 4.x versions
- Upgraded `tar` dep
## 0.6.9
- Upgraded `rc` dep
- Updated known io.js version: v2.4.0
## 0.6.8
- Upgraded `semver` and `rimraf` deps
- Updated known node and io.js versions
## 0.6.7
- Fixed `node_abi` versions for io.js 1.1.x -> 1.8.x (should be 43, but was stored as 42) (refs https://github.com/iojs/build/issues/94)
## 0.6.6
- Updated with known io.js 2.0.0 version
## 0.6.5
- Now respecting `npm_config_node_gyp` (https://github.com/npm/npm/pull/4887)
- Updated to semver@4.3.2
- Updated known node v0.12.x versions and io.js 1.x versions.
## 0.6.4
- Improved support for `io.js` (@fengmk2)
- Test coverage improvements (@mikemorris)
- Fixed support for `--dist-url` that regressed in 0.6.3
## 0.6.3
- Added support for passing raw options to node-gyp using `--` separator. Flags passed after
the `--` to `node-pre-gyp configure` will be passed directly to gyp while flags passed
after the `--` will be passed directly to make/visual studio.
- Added `node-pre-gyp configure` command to be able to call `node-gyp configure` directly
- Fix issue with require validation not working on windows 7 (@edgarsilva)
## 0.6.2
- Support for io.js >= v1.0.2
- Deferred require of `request` and `tar` to help speed up command line usage of `node-pre-gyp`.
## 0.6.1
- Fixed bundled `tar` version
## 0.6.0
- BREAKING: node odd releases like v0.11.x now use `major.minor.patch` for `{node_abi}` instead of `NODE_MODULE_VERSION` (#124)
- Added support for `toolset` option in versioning. By default is an empty string but `--toolset` can be passed to publish or install to select alternative binaries that target a custom toolset like C++11. For example to target Visual Studio 2014 modules like node-sqlite3 use `--toolset=v140`.
- Added support for `--no-rollback` option to request that a failed binary test does not remove the binary module leaves it in place.
- Added support for `--update-binary` option to request an existing binary be re-installed and the check for a valid local module be skipped.
- Added support for passing build options from `npm` through `node-pre-gyp` to `node-gyp`: `--nodedir`, `--disturl`, `--python`, and `--msvs_version`
## 0.5.31
- Added support for deducing node_abi for node.js runtime from previous release if the series is even
- Added support for --target=0.10.33
## 0.5.30
- Repackaged with latest bundled deps
## 0.5.29
- Added support for semver `build`.
- Fixed support for downloading from urls that include `+`.
## 0.5.28
- Now reporting unix style paths only in reveal command
## 0.5.27
- Fixed support for auto-detecting s3 bucket name when it contains `.` - @taavo
- Fixed support for installing when path contains a `'` - @halfdan
- Ported tests to mocha
## 0.5.26
- Fix node-webkit support when `--target` option is not provided
## 0.5.25
- Fix bundling of deps
## 0.5.24
- Updated ABI crosswalk to incldue node v0.10.30 and v0.10.31
## 0.5.23
- Added `reveal` command. Pass no options to get all versioning data as json. Pass a second arg to grab a single versioned property value
- Added support for `--silent` (shortcut for `--loglevel=silent`)
## 0.5.22
- Fixed node-webkit versioning name (NOTE: node-webkit support still experimental)
## 0.5.21
- New package to fix `shasum check failed` error with v0.5.20
## 0.5.20
- Now versioning node-webkit binaries based on major.minor.patch - assuming no compatible ABI across versions (#90)
## 0.5.19
- Updated to know about more node-webkit releases
## 0.5.18
- Updated to know about more node-webkit releases
## 0.5.17
- Updated to know about node v0.10.29 release
## 0.5.16
- Now supporting all aws-sdk configuration parameters (http://docs.aws.amazon.com/AWSJavaScriptSDK/guide/node-configuring.html) (#86)
## 0.5.15
- Fixed installation of windows packages sub directories on unix systems (#84)
## 0.5.14
- Finished support for cross building using `--target_platform` option (#82)
- Now skipping binary validation on install if target arch/platform do not match the host.
- Removed multi-arch validing for OS X since it required a FAT node.js binary
## 0.5.13
- Fix problem in 0.5.12 whereby the wrong versions of mkdirp and semver where bundled.
## 0.5.12
- Improved support for node-webkit (@Mithgol)
## 0.5.11
- Updated target versions listing
## 0.5.10
- Fixed handling of `-debug` flag passed directory to node-pre-gyp (#72)
- Added optional second arg to `node_pre_gyp.find` to customize the default versioning options used to locate the runtime binary
- Failed install due to `testbinary` check failure no longer leaves behind binary (#70)
## 0.5.9
- Fixed regression in `testbinary` command causing installs to fail on windows with 0.5.7 (#60)
## 0.5.8
- Started bundling deps
## 0.5.7
- Fixed the `testbinary` check, which is used to determine whether to re-download or source compile, to work even in complex dependency situations (#63)
- Exposed the internal `testbinary` command in node-pre-gyp command line tool
- Fixed minor bug so that `fallback_to_build` option is always respected
## 0.5.6
- Added support for versioning on the `name` value in `package.json` (#57).
- Moved to using streams for reading tarball when publishing (#52)
## 0.5.5
- Improved binary validation that also now works with node-webkit (@Mithgol)
- Upgraded test apps to work with node v0.11.x
- Improved test coverage
## 0.5.4
- No longer depends on external install of node-gyp for compiling builds.
## 0.5.3
- Reverted fix for debian/nodejs since it broke windows (#45)
## 0.5.2
- Support for debian systems where the node binary is named `nodejs` (#45)
- Added `bin/node-pre-gyp.cmd` to be able to run command on windows locally (npm creates an .npm automatically when globally installed)
- Updated abi-crosswalk with node v0.10.26 entry.
## 0.5.1
- Various minor bug fixes, several improving windows support for publishing.
## 0.5.0
- Changed property names in `binary` object: now required are `module_name`, `module_path`, and `host`.
- Now `module_path` supports versioning, which allows developers to opt-in to using a versioned install path (#18).
- Added `remote_path` which also supports versioning.
- Changed `remote_uri` to `host`.
## 0.4.2
- Added support for `--target` flag to request cross-compile against a specific node/node-webkit version.
- Added preliminary support for node-webkit
- Fixed support for `--target_arch` option being respected in all cases.
## 0.4.1
- Fixed exception when only stderr is available in binary test (@bendi / #31)
## 0.4.0
- Enforce only `https:` based remote publishing access.
- Added `node-pre-gyp info` command to display listing of published binaries
- Added support for changing the directory node-pre-gyp should build in with the `-C/--directory` option.
- Added support for S3 prefixes.
## 0.3.1
- Added `unpublish` command.
- Fixed module path construction in tests.
- Added ability to disable falling back to build behavior via `npm install --fallback-to-build=false` which overrides setting in a depedencies package.json `install` target.
## 0.3.0
- Support for packaging all files in `module_path` directory - see `app4` for example
- Added `testpackage` command.
- Changed `clean` command to only delete `.node` not entire `build` directory since node-gyp will handle that.
- `.node` modules must be in a folder of there own since tar-pack will remove everything when it unpacks.
+27
View File
@@ -0,0 +1,27 @@
Copyright (c), Mapbox
All rights reserved.
Redistribution and use in source and binary forms, with or without modification,
are permitted provided that the following conditions are met:
* Redistributions of source code must retain the above copyright notice,
this list of conditions and the following disclaimer.
* Redistributions in binary form must reproduce the above copyright notice,
this list of conditions and the following disclaimer in the documentation
and/or other materials provided with the distribution.
* Neither the name of node-pre-gyp nor the names of its contributors
may be used to endorse or promote products derived from this software
without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR
CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+585
View File
@@ -0,0 +1,585 @@
# node-pre-gyp
#### node-pre-gyp makes it easy to publish and install Node.js C++ addons from binaries
[![NPM](https://nodei.co/npm/node-pre-gyp.png?downloads=true&downloadRank=true)](https://nodei.co/npm/node-pre-gyp/)
[![Build Status](https://api.travis-ci.org/mapbox/node-pre-gyp.svg)](https://travis-ci.org/mapbox/node-pre-gyp)
[![Build status](https://ci.appveyor.com/api/projects/status/3nxewb425y83c0gv)](https://ci.appveyor.com/project/Mapbox/node-pre-gyp)
[![Dependencies](https://david-dm.org/mapbox/node-pre-gyp.svg)](https://david-dm.org/mapbox/node-pre-gyp)
`node-pre-gyp` stands between [npm](https://github.com/npm/npm) and [node-gyp](https://github.com/Tootallnate/node-gyp) and offers a cross-platform method of binary deployment.
### Features
- A command line tool called `node-pre-gyp` that can install your package's C++ module from a binary.
- A variety of developer targeted commands for packaging, testing, and publishing binaries.
- A JavaScript module that can dynamically require your installed binary: `require('node-pre-gyp').find`
For a hello world example of a module packaged with `node-pre-gyp` see <https://github.com/springmeyer/node-addon-example> and [the wiki ](https://github.com/mapbox/node-pre-gyp/wiki/Modules-using-node-pre-gyp) for real world examples.
## Credits
- The module is modeled after [node-gyp](https://github.com/Tootallnate/node-gyp) by [@Tootallnate](https://github.com/Tootallnate)
- Motivation for initial development came from [@ErisDS](https://github.com/ErisDS) and the [Ghost Project](https://github.com/TryGhost/Ghost).
- Development is sponsored by [Mapbox](https://www.mapbox.com/)
## FAQ
See the [Frequently Ask Questions](https://github.com/mapbox/node-pre-gyp/wiki/FAQ).
## Depends
- Node.js >= node v0.10.x
## Install
`node-pre-gyp` is designed to be installed as a local dependency of your Node.js C++ addon and accessed like:
./node_modules/.bin/node-pre-gyp --help
But you can also install it globally:
npm install node-pre-gyp -g
## Usage
### Commands
View all possible commands:
node-pre-gyp --help
- clean - Remove the entire folder containing the compiled .node module
- install - Install pre-built binary for module
- reinstall - Run "clean" and "install" at once
- build - Compile the module by dispatching to node-gyp or nw-gyp
- rebuild - Run "clean" and "build" at once
- package - Pack binary into tarball
- testpackage - Test that the staged package is valid
- publish - Publish pre-built binary
- unpublish - Unpublish pre-built binary
- info - Fetch info on published binaries
You can also chain commands:
node-pre-gyp clean build unpublish publish info
### Options
Options include:
- `-C/--directory`: run the command in this directory
- `--build-from-source`: build from source instead of using pre-built binary
- `--update-binary`: reinstall by replacing previously installed local binary with remote binary
- `--runtime=node-webkit`: customize the runtime: `node`, `electron` and `node-webkit` are the valid options
- `--fallback-to-build`: fallback to building from source if pre-built binary is not available
- `--target=0.10.25`: Pass the target node or node-webkit version to compile against
- `--target_arch=ia32`: Pass the target arch and override the host `arch`. Valid values are 'ia32','x64', or `arm`.
- `--target_platform=win32`: Pass the target platform and override the host `platform`. Valid values are `linux`, `darwin`, `win32`, `sunos`, `freebsd`, `openbsd`, and `aix`.
Both `--build-from-source` and `--fallback-to-build` can be passed alone or they can provide values. You can pass `--fallback-to-build=false` to override the option as declared in package.json. In addition to being able to pass `--build-from-source` you can also pass `--build-from-source=myapp` where `myapp` is the name of your module.
For example: `npm install --build-from-source=myapp`. This is useful if:
- `myapp` is referenced in the package.json of a larger app and therefore `myapp` is being installed as a dependent with `npm install`.
- The larger app also depends on other modules installed with `node-pre-gyp`
- You only want to trigger a source compile for `myapp` and the other modules.
### Configuring
This is a guide to configuring your module to use node-pre-gyp.
#### 1) Add new entries to your `package.json`
- Add `node-pre-gyp` to `dependencies`
- Add `aws-sdk` as a `devDependency`
- Add a custom `install` script
- Declare a `binary` object
This looks like:
```js
"dependencies" : {
"node-pre-gyp": "0.6.x"
},
"devDependencies": {
"aws-sdk": "2.x"
}
"bundledDependencies":["node-pre-gyp"],
"scripts": {
"install": "node-pre-gyp install --fallback-to-build"
},
"binary": {
"module_name": "your_module",
"module_path": "./lib/binding/",
"host": "https://your_module.s3-us-west-1.amazonaws.com"
}
```
For a full example see [node-addon-examples's package.json](https://github.com/springmeyer/node-addon-example/blob/master/package.json).
Let's break this down:
- Dependencies need to list `node-pre-gyp`
- Your devDependencies should list `aws-sdk` so that you can run `node-pre-gyp publish` locally or a CI system. We recommend using `devDependencies` only since `aws-sdk` is large and not needed for `node-pre-gyp install` since it only uses http to fetch binaries
- You should add `"bundledDependencies":["node-pre-gyp"]`. This ensures that when you publish your module that the correct version of node-pre-gyp will be included in the `node_modules` folder during publishing. Then when uses install your module `node-pre-gyp` will already be present. Without this your module will not be safely installable for downstream applications that have a depedency on node-pre-gyp in the npm tree (without bundling npm deduping might break the install when node-pre-gyp is moved in flight)
- Your `scripts` section should optionally add `"prepublish": "npm ls"` to ensure the right node-pre-gyp version is bundled before publishing your module. If node-pre-gyp is missing or an old version is present then this will catch that error before you publish a broken package.
- Your `scripts` section should override the `install` target with `"install": "node-pre-gyp install --fallback-to-build"`. This allows node-pre-gyp to be used instead of the default npm behavior of always source compiling with `node-gyp` directly.
- Your package.json should contain a `binary` section describing key properties you provide to allow node-pre-gyp to package optimally. They are detailed below.
Note: in the past we recommended using `"preinstall": "npm install node-pre-gyp"` as an alternative method to avoid needing to bundle. But this does not behave predictably across all npm versions - see https://github.com/mapbox/node-pre-gyp/issues/260 for the details. So we do not recommend using `preinstall` to install `node-pre-gyp`. Instead we recommend bundling. More history on this at https://github.com/strongloop/fsevents/issues/157#issuecomment-265545908.
##### The `binary` object has three required properties
###### module_name
The name of your native node module. This value must:
- Match the name passed to [the NODE_MODULE macro](http://nodejs.org/api/addons.html#addons_hello_world)
- Must be a valid C variable name (e.g. it cannot contain `-`)
- Should not include the `.node` extension.
###### module_path
The location your native module is placed after a build. This should be an empty directory without other Javascript files. This entire directory will be packaged in the binary tarball. When installing from a remote package this directory will be overwritten with the contents of the tarball.
Note: This property supports variables based on [Versioning](#versioning).
###### host
A url to the remote location where you've published tarball binaries (must be `https` not `http`).
It is highly recommended that you use Amazon S3. The reasons are:
- Various node-pre-gyp commands like `publish` and `info` only work with an S3 host.
- S3 is a very solid hosting platform for distributing large files.
- We provide detail documentation for using [S3 hosting](#s3-hosting) with node-pre-gyp.
Why then not require S3? Because while some applications using node-pre-gyp need to distribute binaries as large as 20-30 MB, others might have very small binaries and might wish to store them in a GitHub repo. This is not recommended, but if an author really wants to host in a non-s3 location then it should be possible.
It should also be mentioned that there is an optional and entirely separate npm module called [node-pre-gyp-github](https://github.com/bchr02/node-pre-gyp-github) which is intended to complement node-pre-gyp and be installed along with it. It provides the ability to store and publish your binaries within your repositories GitHub Releases if you would rather not use S3 directly. Installation and usage instructions can be found [here](https://github.com/bchr02/node-pre-gyp-github), but the basic premise is that instead of using the ```node-pre-gyp publish``` command you would use ```node-pre-gyp-github publish```.
##### The `binary` object has two optional properties
###### remote_path
It **is recommended** that you customize this property. This is an extra path to use for publishing and finding remote tarballs. The default value for `remote_path` is `""` meaning that if you do not provide it then all packages will be published at the base of the `host`. It is recommended to provide a value like `./{name}/v{version}` to help organize remote packages in the case that you choose to publish multiple node addons to the same `host`.
Note: This property supports variables based on [Versioning](#versioning).
###### package_name
It is **not recommended** to override this property unless you are also overriding the `remote_path`. This is the versioned name of the remote tarball containing the binary `.node` module and any supporting files you've placed inside the `module_path` directory. Unless you specify `package_name` in your `package.json` then it defaults to `{module_name}-v{version}-{node_abi}-{platform}-{arch}.tar.gz` which allows your binary to work across node versions, platforms, and architectures. If you are using `remote_path` that is also versioned by `./{module_name}/v{version}` then you could remove these variables from the `package_name` and just use: `{node_abi}-{platform}-{arch}.tar.gz`. Then your remote tarball will be looked up at, for example, `https://example.com/your-module/v0.1.0/node-v11-linux-x64.tar.gz`.
Avoiding the version of your module in the `package_name` and instead only embedding in a directory name can be useful when you want to make a quick tag of your module that does not change any C++ code. In this case you can just copy binaries to the new version behind the scenes like:
```sh
aws s3 sync --acl public-read s3://mapbox-node-binary/sqlite3/v3.0.3/ s3://mapbox-node-binary/sqlite3/v3.0.4/
```
Note: This property supports variables based on [Versioning](#versioning).
#### 2) Add a new target to binding.gyp
`node-pre-gyp` calls out to `node-gyp` to compile the module and passes variables along like [module_name](#module_name) and [module_path](#module_path).
A new target must be added to `binding.gyp` that moves the compiled `.node` module from `./build/Release/module_name.node` into the directory specified by `module_path`.
Add a target like this at the end of your `targets` list:
```js
{
"target_name": "action_after_build",
"type": "none",
"dependencies": [ "<(module_name)" ],
"copies": [
{
"files": [ "<(PRODUCT_DIR)/<(module_name).node" ],
"destination": "<(module_path)"
}
]
}
```
For a full example see [node-addon-example's binding.gyp](https://github.com/springmeyer/node-addon-example/blob/2ff60a8ded7f042864ad21db00c3a5a06cf47075/binding.gyp).
#### 3) Dynamically require your `.node`
Inside the main js file that requires your addon module you are likely currently doing:
```js
var binding = require('../build/Release/binding.node');
```
or:
```js
var bindings = require('./bindings')
```
Change those lines to:
```js
var binary = require('node-pre-gyp');
var path = require('path');
var binding_path = binary.find(path.resolve(path.join(__dirname,'./package.json')));
var binding = require(binding_path);
```
For a full example see [node-addon-example's index.js](https://github.com/springmeyer/node-addon-example/blob/2ff60a8ded7f042864ad21db00c3a5a06cf47075/index.js#L1-L4)
#### 4) Build and package your app
Now build your module from source:
npm install --build-from-source
The `--build-from-source` tells `node-pre-gyp` to not look for a remote package and instead dispatch to node-gyp to build.
Now `node-pre-gyp` should now also be installed as a local dependency so the command line tool it offers can be found at `./node_modules/.bin/node-pre-gyp`.
#### 5) Test
Now `npm test` should work just as it did before.
#### 6) Publish the tarball
Then package your app:
./node_modules/.bin/node-pre-gyp package
Once packaged, now you can publish:
./node_modules/.bin/node-pre-gyp publish
Currently the `publish` command pushes your binary to S3. This requires:
- You have installed `aws-sdk` with `npm install aws-sdk`
- You have created a bucket already.
- The `host` points to an S3 http or https endpoint.
- You have configured node-pre-gyp to read your S3 credentials (see [S3 hosting](#s3-hosting) for details).
You can also host your binaries elsewhere. To do this requires:
- You manually publish the binary created by the `package` command to an `https` endpoint
- Ensure that the `host` value points to your custom `https` endpoint.
#### 7) Automate builds
Now you need to publish builds for all the platforms and node versions you wish to support. This is best automated.
- See [Appveyor Automation](#appveyor-automation) for how to auto-publish builds on Windows.
- See [Travis Automation](#travis-automation) for how to auto-publish builds on OS X and Linux.
#### 8) You're done!
Now publish your module to the npm registry. Users will now be able to install your module from a binary.
What will happen is this:
1. `npm install <your package>` will pull from the npm registry
2. npm will run the `install` script which will call out to `node-pre-gyp`
3. `node-pre-gyp` will fetch the binary `.node` module and unpack in the right place
4. Assuming that all worked, you are done
If a a binary was not available for a given platform and `--fallback-to-build` was used then `node-gyp rebuild` will be called to try to source compile the module.
## S3 Hosting
You can host wherever you choose but S3 is cheap, `node-pre-gyp publish` expects it, and S3 can be integrated well with [Travis.ci](http://travis-ci.org) to automate builds for OS X and Ubuntu, and with [Appveyor](http://appveyor.com) to automate builds for Windows. Here is an approach to do this:
First, get setup locally and test the workflow:
#### 1) Create an S3 bucket
And have your **key** and **secret key** ready for writing to the bucket.
It is recommended to create a IAM user with a policy that only gives permissions to the specific bucket you plan to publish to. This can be done in the [IAM console](https://console.aws.amazon.com/iam/) by: 1) adding a new user, 2) choosing `Attach User Policy`, 3) Using the `Policy Generator`, 4) selecting `Amazon S3` for the service, 5) adding the actions: `DeleteObject`, `GetObject`, `GetObjectAcl`, `ListBucket`, `PutObject`, `PutObjectAcl`, 6) adding an ARN of `arn:aws:s3:::bucket/*` (replacing `bucket` with your bucket name), and finally 7) clicking `Add Statement` and saving the policy. It should generate a policy like:
```js
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "Stmt1394587197000",
"Effect": "Allow",
"Action": [
"s3:DeleteObject",
"s3:GetObject",
"s3:GetObjectAcl",
"s3:ListBucket",
"s3:PutObject",
"s3:PutObjectAcl"
],
"Resource": [
"arn:aws:s3:::node-pre-gyp-tests/*"
]
}
]
}
```
#### 2) Install node-pre-gyp
Either install it globally:
npm install node-pre-gyp -g
Or put the local version on your PATH
export PATH=`pwd`/node_modules/.bin/:$PATH
#### 3) Configure AWS credentials
There are several ways to do this.
You can use any of the methods described at http://docs.aws.amazon.com/AWSJavaScriptSDK/guide/node-configuring.html.
Or you can create a `~/.node_pre_gyprc`
Or pass options in any way supported by [RC](https://github.com/dominictarr/rc#standards)
A `~/.node_pre_gyprc` looks like:
```js
{
"accessKeyId": "xxx",
"secretAccessKey": "xxx"
}
```
Another way is to use your environment:
export node_pre_gyp_accessKeyId=xxx
export node_pre_gyp_secretAccessKey=xxx
You may also need to specify the `region` if it is not explicit in the `host` value you use. The `bucket` can also be specified but it is optional because `node-pre-gyp` will detect it from the `host` value.
#### 4) Package and publish your build
Install the `aws-sdk`:
npm install aws-sdk
Then publish:
node-pre-gyp package publish
Note: if you hit an error like `Hostname/IP doesn't match certificate's altnames` it may mean that you need to provide the `region` option in your config.
## Appveyor Automation
[Appveyor](http://www.appveyor.com/) can build binaries and publish the results per commit and supports:
- Windows Visual Studio 2013 and related compilers
- Both 64 bit (x64) and 32 bit (x86) build configurations
- Multiple Node.js versions
For an example of doing this see [node-sqlite3's appveyor.yml](https://github.com/mapbox/node-sqlite3/blob/master/appveyor.yml).
Below is a guide to getting set up:
#### 1) Create a free Appveyor account
Go to https://ci.appveyor.com/signup/free and sign in with your GitHub account.
#### 2) Create a new project
Go to https://ci.appveyor.com/projects/new and select the GitHub repo for your module
#### 3) Add appveyor.yml and push it
Once you have committed an `appveyor.yml` ([appveyor.yml reference](http://www.appveyor.com/docs/appveyor-yml)) to your GitHub repo and pushed it AppVeyor should automatically start building your project.
#### 4) Create secure variables
Encrypt your S3 AWS keys by going to <https://ci.appveyor.com/tools/encrypt> and hitting the `encrypt` button.
Then paste the result into your `appveyor.yml`
```yml
environment:
node_pre_gyp_accessKeyId:
secure: Dn9HKdLNYvDgPdQOzRq/DqZ/MPhjknRHB1o+/lVU8MA=
node_pre_gyp_secretAccessKey:
secure: W1rwNoSnOku1r+28gnoufO8UA8iWADmL1LiiwH9IOkIVhDTNGdGPJqAlLjNqwLnL
```
NOTE: keys are per account but not per repo (this is difference than Travis where keys are per repo but not related to the account used to encrypt them).
#### 5) Hook up publishing
Just put `node-pre-gyp package publish` in your `appveyor.yml` after `npm install`.
#### 6) Publish when you want
You might wish to publish binaries only on a specific commit. To do this you could borrow from the [Travis CI idea of commit keywords](http://about.travis-ci.org/docs/user/how-to-skip-a-build/) and add special handling for commit messages with `[publish binary]`:
SET CM=%APPVEYOR_REPO_COMMIT_MESSAGE%
if not "%CM%" == "%CM:[publish binary]=%" node-pre-gyp --msvs_version=2013 publish
If your commit message contains special characters (e.g. `&`) this method might fail. An alternative is to use PowerShell, which gives you additional possibilities, like ignoring case by using `ToLower()`:
ps: if($env:APPVEYOR_REPO_COMMIT_MESSAGE.ToLower().Contains('[publish binary]')) { node-pre-gyp --msvs_version=2013 publish }
Remember this publishing is not the same as `npm publish`. We're just talking about the binary module here and not your entire npm package.
## Travis Automation
[Travis](https://travis-ci.org/) can push to S3 after a successful build and supports both:
- Ubuntu Precise and OS X (64 bit)
- Multiple Node.js versions
For an example of doing this see [node-add-example's .travis.yml](https://github.com/springmeyer/node-addon-example/blob/2ff60a8ded7f042864ad21db00c3a5a06cf47075/.travis.yml).
Note: if you need 32 bit binaries, this can be done from a 64 bit Travis machine. See [the node-sqlite3 scripts for an example of doing this](https://github.com/mapbox/node-sqlite3/blob/bae122aa6a2b8a45f6b717fab24e207740e32b5d/scripts/build_against_node.sh#L54-L74).
Below is a guide to getting set up:
#### 1) Install the Travis gem
gem install travis
#### 2) Create secure variables
Make sure you run this command from within the directory of your module.
Use `travis-encrypt` like:
travis encrypt node_pre_gyp_accessKeyId=${node_pre_gyp_accessKeyId}
travis encrypt node_pre_gyp_secretAccessKey=${node_pre_gyp_secretAccessKey}
Then put those values in your `.travis.yml` like:
```yaml
env:
global:
- secure: F+sEL/v56CzHqmCSSES4pEyC9NeQlkoR0Gs/ZuZxX1ytrj8SKtp3MKqBj7zhIclSdXBz4Ev966Da5ctmcTd410p0b240MV6BVOkLUtkjZJyErMBOkeb8n8yVfSoeMx8RiIhBmIvEn+rlQq+bSFis61/JkE9rxsjkGRZi14hHr4M=
- secure: o2nkUQIiABD139XS6L8pxq3XO5gch27hvm/gOdV+dzNKc/s2KomVPWcOyXNxtJGhtecAkABzaW8KHDDi5QL1kNEFx6BxFVMLO8rjFPsMVaBG9Ks6JiDQkkmrGNcnVdxI/6EKTLHTH5WLsz8+J7caDBzvKbEfTux5EamEhxIWgrI=
```
More details on Travis encryption at http://about.travis-ci.org/docs/user/encryption-keys/.
#### 3) Hook up publishing
Just put `node-pre-gyp package publish` in your `.travis.yml` after `npm install`.
##### OS X publishing
If you want binaries for OS X in addition to linux you can enable [multi-os for Travis](http://docs.travis-ci.com/user/multi-os/#Setting-.travis.yml)
Use a configuration like:
```yml
language: cpp
os:
- linux
- osx
env:
matrix:
- NODE_VERSION="0.10"
- NODE_VERSION="0.11.14"
before_install:
- rm -rf ~/.nvm/ && git clone --depth 1 https://github.com/creationix/nvm.git ~/.nvm
- source ~/.nvm/nvm.sh
- nvm install $NODE_VERSION
- nvm use $NODE_VERSION
```
See [Travis OS X Gotchas](#travis-os-x-gotchas) for why we replace `language: node_js` and `node_js:` sections with `language: cpp` and a custom matrix.
Also create platform specific sections for any deps that need install. For example if you need libpng:
```yml
- if [ $(uname -s) == 'Linux' ]; then apt-get install libpng-dev; fi;
- if [ $(uname -s) == 'Darwin' ]; then brew install libpng; fi;
```
For detailed multi-OS examples see [node-mapnik](https://github.com/mapnik/node-mapnik/blob/master/.travis.yml) and [node-sqlite3](https://github.com/mapbox/node-sqlite3/blob/master/.travis.yml).
##### Travis OS X Gotchas
First, unlike the Travis Linux machines, the OS X machines do not put `node-pre-gyp` on PATH by default. To do so you will need to:
```sh
export PATH=$(pwd)/node_modules/.bin:${PATH}
```
Second, the OS X machines do not support using a matrix for installing different Node.js versions. So you need to bootstrap the installation of Node.js in a cross platform way.
By doing:
```yml
env:
matrix:
- NODE_VERSION="0.10"
- NODE_VERSION="0.11.14"
before_install:
- rm -rf ~/.nvm/ && git clone --depth 1 https://github.com/creationix/nvm.git ~/.nvm
- source ~/.nvm/nvm.sh
- nvm install $NODE_VERSION
- nvm use $NODE_VERSION
```
You can easily recreate the previous behavior of this matrix:
```yml
node_js:
- "0.10"
- "0.11.14"
```
#### 4) Publish when you want
You might wish to publish binaries only on a specific commit. To do this you could borrow from the [Travis CI idea of commit keywords](http://about.travis-ci.org/docs/user/how-to-skip-a-build/) and add special handling for commit messages with `[publish binary]`:
COMMIT_MESSAGE=$(git log --format=%B --no-merges -n 1 | tr -d '\n')
if [[ ${COMMIT_MESSAGE} =~ "[publish binary]" ]]; then node-pre-gyp publish; fi;
Then you can trigger new binaries to be built like:
git commit -a -m "[publish binary]"
Or, if you don't have any changes to make simply run:
git commit --allow-empty -m "[publish binary]"
WARNING: if you are working in a pull request and publishing binaries from there then you will want to avoid double publishing when Travis CI builds both the `push` and `pr`. You only want to run the publish on the `push` commit. See https://github.com/Project-OSRM/node-osrm/blob/8eb837abe2e2e30e595093d16e5354bc5c573575/scripts/is_pr_merge.sh which is called from https://github.com/Project-OSRM/node-osrm/blob/8eb837abe2e2e30e595093d16e5354bc5c573575/scripts/publish.sh for an example of how to do this.
Remember this publishing is not the same as `npm publish`. We're just talking about the binary module here and not your entire npm package. To automate the publishing of your entire package to npm on Travis see http://about.travis-ci.org/docs/user/deployment/npm/
# Versioning
The `binary` properties of `module_path`, `remote_path`, and `package_name` support variable substitution. The strings are evaluated by `node-pre-gyp` depending on your system and any custom build flags you passed.
- `node_abi`: The node C++ `ABI` number. This value is available in Javascript as `process.versions.modules` as of [`>= v0.10.4 >= v0.11.7`](https://github.com/joyent/node/commit/ccabd4a6fa8a6eb79d29bc3bbe9fe2b6531c2d8e) and in C++ as the `NODE_MODULE_VERSION` define much earlier. For versions of Node before this was available we fallback to the V8 major and minor version.
- `platform` matches node's `process.platform` like `linux`, `darwin`, and `win32` unless the user passed the `--target_platform` option to override.
- `arch` matches node's `process.arch` like `x64` or `ia32` unless the user passes the `--target_arch` option to override.
- `configuration` - Either 'Release' or 'Debug' depending on if `--debug` is passed during the build.
- `module_name` - the `binary.module_name` attribute from `package.json`.
- `version` - the semver `version` value for your module from `package.json` (NOTE: ignores the `semver.build` property).
- `major`, `minor`, `patch`, and `prelease` match the individual semver values for your module's `version`
- `build` - the sevmer `build` value. For example it would be `this.that` if your package.json `version` was `v1.0.0+this.that`
- `prerelease` - the semver `prerelease` value. For example it would be `alpha.beta` if your package.json `version` was `v1.0.0-alpha.beta`
The options are visible in the code at <https://github.com/mapbox/node-pre-gyp/blob/612b7bca2604508d881e1187614870ba19a7f0c5/lib/util/versioning.js#L114-L127>
# Download binary files from a mirror
S3 is broken in China for the well known reason.
Using the `npm` config argument: `--{module_name}_binary_host_mirror` can download binary files through a mirror.
e.g.: Install [v8-profiler](https://www.npmjs.com/package/v8-profiler) from `npm`.
```bash
$ npm install v8-profiler --profiler_binary_host_mirror=https://npm.taobao.org/mirrors/node-inspector/
```
+35
View File
@@ -0,0 +1,35 @@
os: Visual Studio 2015
environment:
matrix:
- nodejs_version: 0.10
- nodejs_version: 4
- nodejs_version: 6
platform:
- x64
- x86
shallow_clone: true
install:
- ps: Install-Product node $env:nodejs_version $env:Platform
- ps: Set-ExecutionPolicy Unrestricted -Scope CurrentUser -Force
# https://github.com/nodejs/node-gyp/issues/972#issuecomment-231055109
- npm install --production node-gyp
- node --version
- npm --version
#- SET PATH=%APPDATA%\npm;%PATH%
#- npm install -g node-gyp
- npm install -g node-gyp@3.2.1
- node -e "console.log(process.arch);"
- IF /I "%PLATFORM%" == "x64" set PATH=C:\Python27-x64;%PATH%
- IF /I "%PLATFORM%" == "x86" SET PATH=C:\python27;%PATH%
- IF /I "%PLATFORM%" == "x64" CALL "C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\vcvarsall.bat" amd64
- IF /I "%PLATFORM%" == "x86" CALL "C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\vcvarsall.bat" x86
- npm install
- npm test
build: off
test: off
deploy: off
+131
View File
@@ -0,0 +1,131 @@
#!/usr/bin/env node
"use strict";
/**
* Set the title.
*/
process.title = 'node-pre-gyp';
/**
* Module dependencies.
*/
var node_pre_gyp = require('../');
var log = require('npmlog');
/**
* Process and execute the selected commands.
*/
var prog = new node_pre_gyp.Run();
var completed = false;
prog.parseArgv(process.argv);
if (prog.todo.length === 0) {
if (~process.argv.indexOf('-v') || ~process.argv.indexOf('--version')) {
console.log('v%s', prog.version);
} else {
console.log('%s', prog.usage());
}
return process.exit(0);
}
// if --no-color is passed
if (prog.opts && prog.opts.hasOwnProperty('color') && !prog.opts.color) {
log.disableColor();
}
log.info('it worked if it ends with', 'ok');
log.verbose('cli', process.argv);
log.info('using', process.title + '@%s', prog.version);
log.info('using', 'node@%s | %s | %s', process.versions.node, process.platform, process.arch);
/**
* Change dir if -C/--directory was passed.
*/
var dir = prog.opts.directory;
if (dir) {
var fs = require('fs');
try {
var stat = fs.statSync(dir);
if (stat.isDirectory()) {
log.info('chdir', dir);
process.chdir(dir);
} else {
log.warn('chdir', dir + ' is not a directory');
}
} catch (e) {
if (e.code === 'ENOENT') {
log.warn('chdir', dir + ' is not a directory');
} else {
log.warn('chdir', 'error during chdir() "%s"', e.message);
}
}
}
function run () {
var command = prog.todo.shift();
if (!command) {
// done!
completed = true;
log.info('ok');
return;
}
prog.commands[command.name](command.args, function (err) {
if (err) {
log.error(command.name + ' error');
log.error('stack', err.stack);
errorMessage();
log.error('not ok');
console.log(err.message);
return process.exit(1);
}
var args_array = [].slice.call(arguments, 1);
if (args_array.length) {
console.log.apply(console, args_array);
}
// now run the next command in the queue
process.nextTick(run);
});
}
process.on('exit', function (code) {
if (!completed && !code) {
log.error('Completion callback never invoked!');
issueMessage();
process.exit(6);
}
});
process.on('uncaughtException', function (err) {
log.error('UNCAUGHT EXCEPTION');
log.error('stack', err.stack);
issueMessage();
process.exit(7);
});
function errorMessage () {
// copied from npm's lib/util/error-handler.js
var os = require('os');
log.error('System', os.type() + ' ' + os.release());
log.error('command', process.argv.map(JSON.stringify).join(' '));
log.error('cwd', process.cwd());
log.error('node -v', process.version);
log.error(process.title+' -v', 'v' + prog.package.version);
}
function issueMessage () {
errorMessage();
log.error('', [ 'This is a bug in `'+process.title+'`.',
'Try to update '+process.title+' and file an issue if it does not help:',
' <https://github.com/mapbox/'+process.title+'/issues>',
].join('\n'));
}
// start running the given commands!
run();
+2
View File
@@ -0,0 +1,2 @@
@echo off
node "%~dp0\node-pre-gyp" %*
+73
View File
@@ -0,0 +1,73 @@
{
"AWSTemplateFormatVersion": "2010-09-09",
"Description": "node-pre-gyp bucket resources",
"Resources": {
"BuildUser": {
"Type": "AWS::IAM::User",
"Properties": {
"Policies": [
{
"PolicyName": "list",
"PolicyDocument": {
"Statement": [
{
"Action": [
"s3:ListBucket"
],
"Effect": "Allow",
"Resource": [
"arn:aws:s3:::mason-binaries"
]
}
]
}
},
{
"PolicyName": "build",
"PolicyDocument": {
"Statement": [
{
"Action": [
"s3:DeleteObject",
"s3:GetObject",
"s3:GetObjectAcl",
"s3:PutObject",
"s3:PutObjectAcl"
],
"Effect": "Allow",
"Resource": [
"arn:aws:s3:::mason-binaries/*"
]
}
]
}
}
]
}
},
"BuildUserKey": {
"Type": "AWS::IAM::AccessKey",
"Properties": {
"UserName": {
"Ref": "BuildUser"
}
}
}
},
"Outputs": {
"AccessKeyId": {
"Value": {
"Ref": "BuildUserKey"
}
},
"SecretAccessKey": {
"Value": {
"Fn::GetAtt": [
"BuildUserKey",
"SecretAccessKey"
]
}
}
}
}
+10
View File
@@ -0,0 +1,10 @@
# Contributing
### Releasing a new version:
- Ensure tests are passing on travis (okay for appveyor node v4 x86 to fail)
- Run `node scripts/abi_crosswalk.js` and commit any changes
- Update the changelog
- Tag a new release like: `git tag -a v0.6.34 -m "tagging v0.6.34" && git push --tags`
- Run `npm publish`
+43
View File
@@ -0,0 +1,43 @@
"use strict";
module.exports = exports = build;
exports.usage = 'Attempts to compile the module by dispatching to node-gyp or nw-gyp';
var compile = require('./util/compile.js');
var handle_gyp_opts = require('./util/handle_gyp_opts.js');
var configure = require('./configure.js');
function do_build(gyp,argv,callback) {
handle_gyp_opts(gyp,argv,function(err,result) {
var final_args = ['build'].concat(result.gyp).concat(result.pre);
if (result.unparsed.length > 0) {
final_args = final_args.
concat(['--']).
concat(result.unparsed);
}
compile.run_gyp(final_args,result.opts,function(err) {
return callback(err);
});
});
}
function build(gyp, argv, callback) {
// Form up commands to pass to node-gyp:
// We map `node-pre-gyp build` to `node-gyp configure build` so that we do not
// trigger a clean and therefore do not pay the penalty of a full recompile
if (argv.length && (argv.indexOf('rebuild') > -1)) {
// here we map `node-pre-gyp rebuild` to `node-gyp rebuild` which internally means
// "clean + configure + build" and triggers a full recompile
compile.run_gyp(['clean'],{},function(err) {
if (err) return callback(err);
configure(gyp,argv,function(err) {
if (err) return callback(err);
return do_build(gyp,argv,callback);
});
});
} else {
return do_build(gyp,argv,callback);
}
}
+23
View File
@@ -0,0 +1,23 @@
"use strict";
module.exports = exports = clean;
exports.usage = 'Removes the entire folder containing the compiled .node module';
var fs = require('fs');
var rm = require('rimraf');
var exists = require('fs').exists || require('path').exists;
var versioning = require('./util/versioning.js');
function clean (gyp, argv, callback) {
var package_json = JSON.parse(fs.readFileSync('./package.json'));
var opts = versioning.evaluate(package_json, gyp.opts);
var to_delete = opts.module_path;
exists(to_delete, function(found) {
if (found) {
if (!gyp.opts.silent_clean) console.log('['+package_json.name+'] Removing "%s"', to_delete);
return rm(to_delete, callback);
}
return callback();
});
}
+48
View File
@@ -0,0 +1,48 @@
"use strict";
module.exports = exports = configure;
exports.usage = 'Attempts to configure node-gyp or nw-gyp build';
var compile = require('./util/compile.js');
var handle_gyp_opts = require('./util/handle_gyp_opts.js');
function configure(gyp, argv, callback) {
handle_gyp_opts(gyp,argv,function(err,result) {
var final_args = result.gyp.concat(result.pre);
// pull select node-gyp configure options out of the npm environ
var known_gyp_args = ['dist-url','python','nodedir','msvs_version'];
known_gyp_args.forEach(function(key) {
var val = gyp.opts[key] || gyp.opts[key.replace('-','_')];
if (val) {
final_args.push('--'+key+'='+val);
}
});
// --ensure=false tell node-gyp to re-install node development headers
// but it is only respected by node-gyp install, so we have to call install
// as a separate step if the user passes it
if (gyp.opts.ensure === false) {
var install_args = final_args.concat(['install','--ensure=false']);
compile.run_gyp(install_args,result.opts,function(err) {
if (err) return callback(err);
if (result.unparsed.length > 0) {
final_args = final_args.
concat(['--']).
concat(result.unparsed);
}
compile.run_gyp(['configure'].concat(final_args),result.opts,function(err) {
return callback(err);
});
});
} else {
if (result.unparsed.length > 0) {
final_args = final_args.
concat(['--']).
concat(result.unparsed);
}
compile.run_gyp(['configure'].concat(final_args),result.opts,function(err) {
return callback(err);
});
}
});
}
+40
View File
@@ -0,0 +1,40 @@
"use strict";
module.exports = exports = unpublish;
exports.usage = 'Lists all published binaries (requires aws-sdk)';
var fs = require('fs');
var log = require('npmlog');
var versioning = require('./util/versioning.js');
var s3_setup = require('./util/s3_setup.js');
var config = require('rc')("node_pre_gyp",{acl:"public-read"});
function unpublish(gyp, argv, callback) {
var AWS = require("aws-sdk");
var package_json = JSON.parse(fs.readFileSync('./package.json'));
var opts = versioning.evaluate(package_json, gyp.opts);
s3_setup.detect(opts.hosted_path,config);
AWS.config.update(config);
var s3 = new AWS.S3();
var s3_opts = { Bucket: config.bucket,
Prefix: config.prefix
};
s3.listObjects(s3_opts, function(err, meta){
if (err && err.code == 'NotFound') {
return callback(new Error('['+package_json.name+'] Not found: https://' + s3_opts.Bucket + '.s3.amazonaws.com/'+config.prefix));
} else if(err) {
return callback(err);
} else {
log.verbose(JSON.stringify(meta,null,1));
if (meta && meta.Contents) {
meta.Contents.forEach(function(obj) {
console.log(obj.Key);
});
} else {
console.error('['+package_json.name+'] No objects found at https://' + s3_opts.Bucket + '.s3.amazonaws.com/'+config.prefix );
}
return callback();
}
});
}
+197
View File
@@ -0,0 +1,197 @@
"use strict";
module.exports = exports = install;
exports.usage = 'Attempts to install pre-built binary for module';
var fs = require('fs');
var path = require('path');
var zlib = require('zlib');
var log = require('npmlog');
var existsAsync = fs.exists || path.exists;
var versioning = require('./util/versioning.js');
var npgVersion = 'unknown';
try {
// Read own package.json to get the current node-pre-pyp version.
var ownPackageJSON = fs.readFileSync(path.join(__dirname, '..', 'package.json'), 'utf8');
npgVersion = JSON.parse(ownPackageJSON).version;
} catch (e) {}
function download(uri,opts,callback) {
log.http('GET', uri);
var req = null;
// Try getting version info from the currently running npm.
var envVersionInfo = process.env.npm_config_user_agent ||
'node ' + process.version;
var requestOpts = {
uri: uri.replace('+','%2B'),
headers: {
'User-Agent': 'node-pre-gyp (v' + npgVersion + ', ' + envVersionInfo + ')'
}
};
if (opts.cafile) {
try {
requestOpts.ca = fs.readFileSync(opts.cafile);
} catch (e) {
return callback(e);
}
} else if (opts.ca) {
requestOpts.ca = opts.ca;
}
var proxyUrl = opts.proxy ||
process.env.http_proxy ||
process.env.HTTP_PROXY ||
process.env.npm_config_proxy;
if (proxyUrl) {
if (/^https?:\/\//i.test(proxyUrl)) {
log.verbose('download', 'using proxy url: "%s"', proxyUrl);
requestOpts.proxy = proxyUrl;
} else {
log.warn('download', 'ignoring invalid "proxy" config setting: "%s"', proxyUrl);
}
}
try {
req = require('request')(requestOpts);
} catch (e) {
return callback(e);
}
if (req) {
req.on('response', function (res) {
log.http(res.statusCode, uri);
});
}
return callback(null,req);
}
function place_binary(from,to,opts,callback) {
download(from,opts,function(err,req) {
if (err) return callback(err);
if (!req) return callback(new Error("empty req"));
var badDownload = false;
var extractCount = 0;
var gunzip = zlib.createGunzip();
var extracter = require('tar').Extract({ path: to, strip: 1});
function afterTarball(err) {
if (err) return callback(err);
if (badDownload) return callback(new Error("bad download"));
if (extractCount === 0) {
return callback(new Error('There was a fatal problem while downloading/extracting the tarball'));
}
log.info('tarball', 'done parsing tarball');
callback();
}
function filter_func(entry) {
// ensure directories are +x
// https://github.com/mapnik/node-mapnik/issues/262
entry.props.mode |= (entry.props.mode >>> 2) & parseInt('0111',8);
log.info('install','unpacking ' + entry.path);
extractCount++;
}
gunzip.on('error', callback);
extracter.on('entry', filter_func);
extracter.on('error', callback);
extracter.on('end', afterTarball);
req.on('error', function(err) {
badDownload = true;
return callback(err);
});
req.on('close', function () {
if (extractCount === 0) {
return callback(new Error('Connection closed while downloading tarball file'));
}
});
req.on('response', function(res) {
if (res.statusCode !== 200) {
badDownload = true;
var err = new Error(res.statusCode + ' status code downloading tarball ' + from);
err.statusCode = res.statusCode;
return callback(err);
}
// start unzipping and untaring
req.pipe(gunzip).pipe(extracter);
});
});
}
function do_build(gyp,argv,callback) {
gyp.todo.push( { name: 'build', args: ['rebuild'] } );
process.nextTick(callback);
}
function print_fallback_error(err,opts,package_json) {
var fallback_message = ' (falling back to source compile with node-gyp)';
var full_message = "Pre-built binaries not found for " + package_json.name + "@" + package_json.version;
full_message += " and " + opts.runtime + "@" + (opts.target || process.versions.node) + " (" + opts.node_abi + " ABI)";
full_message += fallback_message;
log.error("Tried to download(" + err.statusCode + "): " + opts.hosted_tarball);
log.error(full_message);
log.http(err.message);
}
function install(gyp, argv, callback) {
var package_json = JSON.parse(fs.readFileSync('./package.json'));
var source_build = gyp.opts['build-from-source'] || gyp.opts.build_from_source;
var update_binary = gyp.opts['update-binary'] || gyp.opts.update_binary;
var should_do_source_build = source_build === package_json.name || (source_build === true || source_build === 'true');
if (should_do_source_build) {
log.info('build','requesting source compile');
return do_build(gyp,argv,callback);
} else {
var fallback_to_build = gyp.opts['fallback-to-build'] || gyp.opts.fallback_to_build;
var should_do_fallback_build = fallback_to_build === package_json.name || (fallback_to_build === true || fallback_to_build === 'true');
// but allow override from npm
if (process.env.npm_config_argv) {
var cooked = JSON.parse(process.env.npm_config_argv).cooked;
var match = cooked.indexOf("--fallback-to-build");
if (match > -1 && cooked.length > match && cooked[match+1] == "false") {
should_do_fallback_build = false;
log.info('install','Build fallback disabled via npm flag: --fallback-to-build=false');
}
}
var opts;
try {
opts = versioning.evaluate(package_json, gyp.opts);
} catch (err) {
return callback(err);
}
opts.ca = gyp.opts.ca;
opts.cafile = gyp.opts.cafile;
var from = opts.hosted_tarball;
var to = opts.module_path;
var binary_module = path.join(to,opts.module_name + '.node');
if (existsAsync(binary_module,function(found) {
if (found && !update_binary) {
console.log('['+package_json.name+'] Success: "' + binary_module + '" already installed');
console.log('Pass --update-binary to reinstall or --build-from-source to recompile');
return callback();
} else {
if (!update_binary) log.info('check','checked for "' + binary_module + '" (not found)');
place_binary(from,to,opts,function(err) {
if (err && should_do_fallback_build) {
print_fallback_error(err,opts,package_json);
return do_build(gyp,argv,callback);
} else if (err) {
return callback(err);
} else {
console.log('['+package_json.name+'] Success: "' + binary_module + '" is installed via remote');
return callback();
}
});
}
}));
}
}
+194
View File
@@ -0,0 +1,194 @@
"use strict";
/**
* Module exports.
*/
module.exports = exports;
/**
* Module dependencies.
*/
var path = require('path');
var nopt = require('nopt');
var log = require('npmlog');
log.disableProgress();
var EE = require('events').EventEmitter;
var inherits = require('util').inherits;
var commands = [
'clean',
'install',
'reinstall',
'build',
'rebuild',
'package',
'testpackage',
'publish',
'unpublish',
'info',
'testbinary',
'reveal',
'configure'
];
var aliases = {};
// differentiate node-pre-gyp's logs from npm's
log.heading = 'node-pre-gyp';
exports.find = require('./pre-binding').find;
function Run() {
var self = this;
this.commands = {};
commands.forEach(function (command) {
self.commands[command] = function (argv, callback) {
log.verbose('command', command, argv);
return require('./' + command)(self, argv, callback);
};
});
}
inherits(Run, EE);
exports.Run = Run;
var proto = Run.prototype;
/**
* Export the contents of the package.json.
*/
proto.package = require('../package.json');
/**
* nopt configuration definitions
*/
proto.configDefs = {
help: Boolean, // everywhere
arch: String, // 'configure'
debug: Boolean, // 'build'
directory: String, // bin
proxy: String, // 'install'
loglevel: String, // everywhere
};
/**
* nopt shorthands
*/
proto.shorthands = {
release: '--no-debug',
C: '--directory',
debug: '--debug',
j: '--jobs',
silent: '--loglevel=silent',
silly: '--loglevel=silly',
verbose: '--loglevel=verbose',
};
/**
* expose the command aliases for the bin file to use.
*/
proto.aliases = aliases;
/**
* Parses the given argv array and sets the 'opts',
* 'argv' and 'command' properties.
*/
proto.parseArgv = function parseOpts (argv) {
this.opts = nopt(this.configDefs, this.shorthands, argv);
this.argv = this.opts.argv.remain.slice();
var commands = this.todo = [];
// create a copy of the argv array with aliases mapped
argv = this.argv.map(function (arg) {
// is this an alias?
if (arg in this.aliases) {
arg = this.aliases[arg];
}
return arg;
}, this);
// process the mapped args into "command" objects ("name" and "args" props)
argv.slice().forEach(function (arg) {
if (arg in this.commands) {
var args = argv.splice(0, argv.indexOf(arg));
argv.shift();
if (commands.length > 0) {
commands[commands.length - 1].args = args;
}
commands.push({ name: arg, args: [] });
}
}, this);
if (commands.length > 0) {
commands[commands.length - 1].args = argv.splice(0);
}
// support for inheriting config env variables from npm
var npm_config_prefix = 'npm_config_';
Object.keys(process.env).forEach(function (name) {
if (name.indexOf(npm_config_prefix) !== 0) return;
var val = process.env[name];
if (name === npm_config_prefix + 'loglevel') {
log.level = val;
} else {
// add the user-defined options to the config
name = name.substring(npm_config_prefix.length);
// avoid npm argv clobber already present args
// which avoids problem of 'npm test' calling
// script that runs unique npm install commands
if (name === 'argv') {
if (this.opts.argv &&
this.opts.argv.remain &&
this.opts.argv.remain.length) {
// do nothing
} else {
this.opts[name] = val;
}
} else {
this.opts[name] = val;
}
}
}, this);
if (this.opts.loglevel) {
log.level = this.opts.loglevel;
}
log.resume();
};
/**
* Returns the usage instructions for node-pre-gyp.
*/
proto.usage = function usage () {
var str = [
'',
' Usage: node-pre-gyp <command> [options]',
'',
' where <command> is one of:',
commands.map(function (c) {
return ' - ' + c + ' - ' + require('./' + c).usage;
}).join('\n'),
'',
'node-pre-gyp@' + this.version + ' ' + path.resolve(__dirname, '..'),
'node@' + process.versions.node
].join('\n');
return str;
};
/**
* Version number getter.
*/
Object.defineProperty(proto, 'version', {
get: function () {
return this.package.version;
},
enumerable: true
});
+46
View File
@@ -0,0 +1,46 @@
"use strict";
module.exports = exports = _package;
exports.usage = 'Packs binary (and enclosing directory) into locally staged tarball';
var fs = require('fs');
var path = require('path');
var log = require('npmlog');
var versioning = require('./util/versioning.js');
var write = require('fs').createWriteStream;
var existsAsync = fs.exists || path.exists;
var mkdirp = require('mkdirp');
function _package(gyp, argv, callback) {
var pack = require('tar-pack').pack;
var package_json = JSON.parse(fs.readFileSync('./package.json'));
var opts = versioning.evaluate(package_json, gyp.opts);
var from = opts.module_path;
var binary_module = path.join(from,opts.module_name + '.node');
existsAsync(binary_module,function(found) {
if (!found) {
return callback(new Error("Cannot package because " + binary_module + " missing: run `node-pre-gyp rebuild` first"));
}
var tarball = opts.staged_tarball;
var filter_func = function(entry) {
// ensure directories are +x
// https://github.com/mapnik/node-mapnik/issues/262
log.info('package','packing ' + entry.path);
return true;
};
mkdirp(path.dirname(tarball),function(err) {
if (err) throw err;
pack(from, { filter: filter_func })
.pipe(write(tarball))
.on('error', function(err) {
if (err) console.error('['+package_json.name+'] ' + err.message);
return callback(err);
})
.on('close', function() {
log.info('package','Binary staged at "' + tarball + '"');
return callback();
});
});
});
}
+25
View File
@@ -0,0 +1,25 @@
"use strict";
var versioning = require('../lib/util/versioning.js');
var existsSync = require('fs').existsSync || require('path').existsSync;
var path = require('path');
module.exports = exports;
exports.usage = 'Finds the require path for the node-pre-gyp installed module';
exports.validate = function(package_json) {
versioning.validate_config(package_json);
};
exports.find = function(package_json_path,opts) {
if (!existsSync(package_json_path)) {
throw new Error("package.json does not exist at " + package_json_path);
}
var package_json = require(package_json_path);
versioning.validate_config(package_json);
opts = opts || {};
if (!opts.module_root) opts.module_root = path.dirname(package_json_path);
var meta = versioning.evaluate(package_json,opts);
return meta.module;
};
+77
View File
@@ -0,0 +1,77 @@
"use strict";
module.exports = exports = publish;
exports.usage = 'Publishes pre-built binary (requires aws-sdk)';
var fs = require('fs');
var path = require('path');
var log = require('npmlog');
var versioning = require('./util/versioning.js');
var s3_setup = require('./util/s3_setup.js');
var existsAsync = fs.exists || path.exists;
var url = require('url');
var config = require('rc')("node_pre_gyp",{acl:"public-read"});
function publish(gyp, argv, callback) {
var AWS = require("aws-sdk");
var package_json = JSON.parse(fs.readFileSync('./package.json'));
var opts = versioning.evaluate(package_json, gyp.opts);
var tarball = opts.staged_tarball;
existsAsync(tarball,function(found) {
if (!found) {
return callback(new Error("Cannot publish because " + tarball + " missing: run `node-pre-gyp package` first"));
}
log.info('publish', 'Detecting s3 credentials');
s3_setup.detect(opts.hosted_path,config);
var key_name = url.resolve(config.prefix,opts.package_name);
log.info('publish', 'Authenticating with s3');
AWS.config.update(config);
var s3 = new AWS.S3();
var s3_opts = { Bucket: config.bucket,
Key: key_name
};
var remote_package = 'https://' + s3_opts.Bucket + '.s3.amazonaws.com/' + s3_opts.Key;
log.info('publish', 'Checking for existing binary at ' + remote_package);
s3.headObject(s3_opts, function(err, meta){
if (meta) log.info('publish', JSON.stringify(meta));
if (err && err.code == 'NotFound') {
// we are safe to publish because
// the object does not already exist
log.info('publish', 'Preparing to put object');
var s3_put = new AWS.S3();
var s3_put_opts = { ACL: config.acl,
Body: fs.createReadStream(tarball),
Bucket: config.bucket,
Key: key_name
};
log.info('publish', 'Putting object');
try {
s3_put.putObject(s3_put_opts, function(err, resp){
log.info('publish', 'returned from putting object');
if(err) {
log.info('publish', 's3 putObject error: "' + err + '"');
return callback(err);
}
if (resp) log.info('publish', 's3 putObject response: "' + JSON.stringify(resp) + '"');
log.info('publish', 'successfully put object');
console.log('['+package_json.name+'] published to ' + remote_package);
return callback();
});
} catch (err) {
log.info('publish', 's3 putObject error: "' + err + '"');
return callback(err);
}
} else if (err) {
log.info('publish', 's3 headObject error: "' + err + '"');
return callback(err);
} else {
log.error('publish','Cannot publish over existing version');
log.error('publish',"Update the 'version' field in package.json and try again");
log.error('publish','If the previous version was published in error see:');
log.error('publish','\t node-pre-gyp unpublish');
return callback(new Error('Failed publishing to ' + remote_package));
}
});
});
}
+13
View File
@@ -0,0 +1,13 @@
"use strict";
module.exports = exports = rebuild;
exports.usage = 'Runs "clean" and "build" at once';
function rebuild (gyp, argv, callback) {
gyp.todo.unshift(
{ name: 'clean', args: [] },
{ name: 'build', args: ['rebuild'] }
);
process.nextTick(callback);
}
+13
View File
@@ -0,0 +1,13 @@
"use strict";
module.exports = exports = rebuild;
exports.usage = 'Runs "clean" and "install" at once';
function rebuild (gyp, argv, callback) {
gyp.todo.unshift(
{ name: 'clean', args: [] },
{ name: 'install', args: [] }
);
process.nextTick(callback);
}
+31
View File
@@ -0,0 +1,31 @@
"use strict";
module.exports = exports = reveal;
exports.usage = 'Reveals data on the versioned binary';
var fs = require('fs');
var versioning = require('./util/versioning.js');
function unix_paths(key, val) {
return val && val.replace ? val.replace(/\\/g, '/') : val;
}
function reveal(gyp, argv, callback) {
var package_json = JSON.parse(fs.readFileSync('./package.json'));
var opts = versioning.evaluate(package_json, gyp.opts);
var hit = false;
// if a second arg is passed look to see
// if it is a known option
//console.log(JSON.stringify(gyp.opts,null,1))
var remain = gyp.opts.argv.remain.pop();
if (remain && opts.hasOwnProperty(remain)) {
console.log(opts[remain].replace(/\\/g, '/'));
hit = true;
}
// otherwise return all options as json
if (!hit) {
console.log(JSON.stringify(opts,unix_paths,2));
}
return callback();
}
+79
View File
@@ -0,0 +1,79 @@
"use strict";
module.exports = exports = testbinary;
exports.usage = 'Tests that the binary.node can be required';
var fs = require('fs');
var path = require('path');
var log = require('npmlog');
var cp = require('child_process');
var versioning = require('./util/versioning.js');
var path = require('path');
function testbinary(gyp, argv, callback) {
var args = [];
var options = {};
var shell_cmd = process.execPath;
var package_json = JSON.parse(fs.readFileSync('./package.json'));
var opts = versioning.evaluate(package_json, gyp.opts);
// skip validation for runtimes we don't explicitly support (like electron)
if (opts.runtime &&
opts.runtime !== 'node-webkit' &&
opts.runtime !== 'node') {
return callback();
}
var nw = (opts.runtime && opts.runtime === 'node-webkit');
// ensure on windows that / are used for require path
var binary_module = opts.module.replace(/\\/g, '/');
if ((process.arch != opts.target_arch) ||
(process.platform != opts.target_platform)) {
var msg = "skipping validation since host platform/arch (";
msg += process.platform+'/'+process.arch+")";
msg += " does not match target (";
msg += opts.target_platform+'/'+opts.target_arch+")";
log.info('validate', msg);
return callback();
}
if (nw) {
options.timeout = 5000;
if (process.platform === 'darwin') {
shell_cmd = 'node-webkit';
} else if (process.platform === 'win32') {
shell_cmd = 'nw.exe';
} else {
shell_cmd = 'nw';
}
var modulePath = path.resolve(binary_module);
var appDir = path.join(__dirname, 'util', 'nw-pre-gyp');
args.push(appDir);
args.push(modulePath);
log.info("validate","Running test command: '" + shell_cmd + ' ' + args.join(' ') + "'");
cp.execFile(shell_cmd, args, options, function(err, stdout, stderr) {
// check for normal timeout for node-webkit
if (err) {
if (err.killed === true && err.signal && err.signal.indexOf('SIG') > -1) {
return callback();
}
var stderrLog = stderr.toString();
log.info('stderr', stderrLog);
if( /^\s*Xlib:\s*extension\s*"RANDR"\s*missing\s*on\s*display\s*":\d+\.\d+"\.\s*$/.test(stderrLog) ){
log.info('RANDR', 'stderr contains only RANDR error, ignored');
return callback();
}
return callback(err);
}
return callback();
});
return;
}
args.push('--eval');
args.push("require('" + binary_module.replace(/'/g, '\'') +"')");
log.info("validate","Running test command: '" + shell_cmd + ' ' + args.join(' ') + "'");
cp.execFile(shell_cmd, args, options, function(err, stdout, stderr) {
if (err) {
return callback(err, { stdout:stdout, stderr:stderr});
}
return callback();
});
}
+49
View File
@@ -0,0 +1,49 @@
"use strict";
module.exports = exports = testpackage;
exports.usage = 'Tests that the staged package is valid';
var fs = require('fs');
var path = require('path');
var log = require('npmlog');
var existsAsync = fs.exists || path.exists;
var versioning = require('./util/versioning.js');
var testbinary = require('./testbinary.js');
var read = require('fs').createReadStream;
var zlib = require('zlib');
function testpackage(gyp, argv, callback) {
var package_json = JSON.parse(fs.readFileSync('./package.json'));
var opts = versioning.evaluate(package_json, gyp.opts);
var tarball = opts.staged_tarball;
existsAsync(tarball, function(found) {
if (!found) {
return callback(new Error("Cannot test package because " + tarball + " missing: run `node-pre-gyp package` first"));
}
var to = opts.module_path;
var gunzip = zlib.createGunzip();
var extracter = require('tar').Extract({ path: to, strip: 1 });
function filter_func(entry) {
// ensure directories are +x
// https://github.com/mapnik/node-mapnik/issues/262
entry.props.mode |= (entry.props.mode >>> 2) & parseInt('0111',8);
log.info('install','unpacking ' + entry.path);
}
gunzip.on('error', callback);
extracter.on('error', callback);
extracter.on('entry', filter_func);
extracter.on('end', function(err) {
if (err) return callback(err);
testbinary(gyp,argv,function(err) {
if (err) {
return callback(err);
} else {
console.log('['+package_json.name+'] Package appears valid');
return callback();
}
});
});
read(tarball).pipe(gunzip).pipe(extracter);
});
}
+41
View File
@@ -0,0 +1,41 @@
"use strict";
module.exports = exports = unpublish;
exports.usage = 'Unpublishes pre-built binary (requires aws-sdk)';
var fs = require('fs');
var log = require('npmlog');
var versioning = require('./util/versioning.js');
var s3_setup = require('./util/s3_setup.js');
var url = require('url');
var config = require('rc')("node_pre_gyp",{acl:"public-read"});
function unpublish(gyp, argv, callback) {
var AWS = require("aws-sdk");
var package_json = JSON.parse(fs.readFileSync('./package.json'));
var opts = versioning.evaluate(package_json, gyp.opts);
s3_setup.detect(opts.hosted_path,config);
AWS.config.update(config);
var key_name = url.resolve(config.prefix,opts.package_name);
var s3 = new AWS.S3();
var s3_opts = { Bucket: config.bucket,
Key: key_name
};
s3.headObject(s3_opts, function(err, meta) {
if (err && err.code == 'NotFound') {
console.log('['+package_json.name+'] Not found: https://' + s3_opts.Bucket + '.s3.amazonaws.com/' + s3_opts.Key);
return callback();
} else if(err) {
return callback(err);
} else {
log.info('unpublish', JSON.stringify(meta));
s3.deleteObject(s3_opts, function(err, resp) {
if (err) return callback(err);
log.info(JSON.stringify(resp));
console.log('['+package_json.name+'] Success: removed https://' + s3_opts.Bucket + '.s3.amazonaws.com/' + s3_opts.Key);
return callback();
});
}
});
}
File diff suppressed because it is too large Load Diff
+87
View File
@@ -0,0 +1,87 @@
"use strict";
module.exports = exports;
var fs = require('fs');
var path = require('path');
var win = process.platform == 'win32';
var existsSync = fs.existsSync || path.existsSync;
var cp = require('child_process');
// try to build up the complete path to node-gyp
/* priority:
- node-gyp on ENV:npm_config_node_gyp (https://github.com/npm/npm/pull/4887)
- node-gyp on NODE_PATH
- node-gyp inside npm on NODE_PATH (ignore on iojs)
- node-gyp inside npm beside node exe
*/
function which_node_gyp() {
var node_gyp_bin;
if (process.env.npm_config_node_gyp) {
try {
node_gyp_bin = process.env.npm_config_node_gyp;
if (existsSync(node_gyp_bin)) {
return node_gyp_bin;
}
} catch (err) { }
}
try {
var node_gyp_main = require.resolve('node-gyp');
node_gyp_bin = path.join(path.dirname(
path.dirname(node_gyp_main)),
'bin/node-gyp.js');
if (existsSync(node_gyp_bin)) {
return node_gyp_bin;
}
} catch (err) { }
if (process.execPath.indexOf('iojs') === -1) {
try {
var npm_main = require.resolve('npm');
node_gyp_bin = path.join(path.dirname(
path.dirname(npm_main)),
'node_modules/node-gyp/bin/node-gyp.js');
if (existsSync(node_gyp_bin)) {
return node_gyp_bin;
}
} catch (err) { }
}
var npm_base = path.join(path.dirname(
path.dirname(process.execPath)),
'lib/node_modules/npm/');
node_gyp_bin = path.join(npm_base, 'node_modules/node-gyp/bin/node-gyp.js');
if (existsSync(node_gyp_bin)) {
return node_gyp_bin;
}
}
module.exports.run_gyp = function(args,opts,callback) {
var shell_cmd = '';
var cmd_args = [];
if (opts.runtime && opts.runtime == 'node-webkit') {
shell_cmd = 'nw-gyp';
if (win) shell_cmd += '.cmd';
} else {
var node_gyp_path = which_node_gyp();
if (node_gyp_path) {
shell_cmd = process.execPath;
cmd_args.push(node_gyp_path);
} else {
shell_cmd = 'node-gyp';
if (win) shell_cmd += '.cmd';
}
}
var final_args = cmd_args.concat(args);
var cmd = cp.spawn(shell_cmd, final_args, {cwd: undefined, env: process.env, stdio: [ 0, 1, 2]});
cmd.on('error', function (err) {
if (err) {
return callback(new Error("Failed to execute '" + shell_cmd + ' ' + final_args.join(' ') + "' (" + err + ")"));
}
callback(null,opts);
});
cmd.on('close', function (code) {
if (code && code !== 0) {
return callback(new Error("Failed to execute '" + shell_cmd + ' ' + final_args.join(' ') + "' (" + code + ")"));
}
callback(null,opts);
});
};
+94
View File
@@ -0,0 +1,94 @@
"use strict";
module.exports = exports = handle_gyp_opts;
var fs = require('fs');
var versioning = require('./versioning.js');
/*
Here we gather node-pre-gyp generated options (from versioning) and pass them along to node-gyp.
We massage the args and options slightly to account for differences in what commands mean between
node-pre-gyp and node-gyp (e.g. see the difference between "build" and "rebuild" below)
Keep in mind: the values inside `argv` and `gyp.opts` below are different depending on whether
node-pre-gyp is called directory, or if it is called in a `run-script` phase of npm.
We also try to preserve any command line options that might have been passed to npm or node-pre-gyp.
But this is fairly difficult without passing way to much through. For example `gyp.opts` contains all
the process.env and npm pushes a lot of variables into process.env which node-pre-gyp inherits. So we have
to be very selective about what we pass through.
For example:
`npm install --build-from-source` will give:
argv == [ 'rebuild' ]
gyp.opts.argv == { remain: [ 'install' ],
cooked: [ 'install', '--fallback-to-build' ],
original: [ 'install', '--fallback-to-build' ] }
`./bin/node-pre-gyp build` will give:
argv == []
gyp.opts.argv == { remain: [ 'build' ],
cooked: [ 'build' ],
original: [ '-C', 'test/app1', 'build' ] }
*/
// select set of node-pre-gyp versioning info
// to share with node-gyp
var share_with_node_gyp = [
'module',
'module_name',
'module_path',
];
function handle_gyp_opts(gyp, argv, callback) {
// Collect node-pre-gyp specific variables to pass to node-gyp
var node_pre_gyp_options = [];
// generate custom node-pre-gyp versioning info
var opts = versioning.evaluate(JSON.parse(fs.readFileSync('./package.json')), gyp.opts);
share_with_node_gyp.forEach(function(key) {
var val = opts[key];
if (val) {
node_pre_gyp_options.push('--' + key + '=' + val);
} else {
return callback(new Error("Option " + key + " required but not found by node-pre-gyp"));
}
});
// Collect options that follow the special -- which disables nopt parsing
var unparsed_options = [];
var double_hyphen_found = false;
gyp.opts.argv.original.forEach(function(opt) {
if (double_hyphen_found) {
unparsed_options.push(opt);
}
if (opt == '--') {
double_hyphen_found = true;
}
});
// We try respect and pass through remaining command
// line options (like --foo=bar) to node-gyp
var cooked = gyp.opts.argv.cooked;
var node_gyp_options = [];
cooked.forEach(function(value) {
if (value.length > 2 && value.slice(0,2) == '--') {
var key = value.slice(2);
var val = cooked[cooked.indexOf(value)+1];
if (val && val.indexOf('--') === -1) { // handle '--foo=bar' or ['--foo','bar']
node_gyp_options.push('--' + key + '=' + val);
} else { // pass through --foo
node_gyp_options.push(value);
}
}
});
var result = {'opts':opts,'gyp':node_gyp_options,'pre':node_pre_gyp_options,'unparsed':unparsed_options};
return callback(null,result);
}
+26
View File
@@ -0,0 +1,26 @@
<!doctype html>
<html>
<head>
<meta charset="utf-8">
<title>Node-webkit-based module test</title>
<script>
function nwModuleTest(){
var util = require('util');
var moduleFolder = require('nw.gui').App.argv[0];
try {
require(moduleFolder);
} catch(e) {
if( process.platform !== 'win32' ){
util.log('nw-pre-gyp error:');
util.log(e.stack);
}
process.exit(1);
}
process.exit(0);
}
</script>
</head>
<body onload="nwModuleTest()">
<h1>Node-webkit-based module test</h1>
</body>
</html>
@@ -0,0 +1,9 @@
{
"main": "index.html",
"name": "nw-pre-gyp-module-test",
"description": "Node-webkit-based module test.",
"version": "0.0.1",
"window": {
"show": false
}
}
+27
View File
@@ -0,0 +1,27 @@
"use strict";
module.exports = exports;
var url = require('url');
var URI_REGEX="^(.*)\.(s3(?:-.*)?)\.amazonaws\.com$";
module.exports.detect = function(to,config) {
var uri = url.parse(to);
var hostname_matches = uri.hostname.match(URI_REGEX);
config.prefix = (!uri.pathname || uri.pathname == '/') ? '' : uri.pathname.replace('/','');
if(!hostname_matches) {
return;
}
if (!config.bucket) {
config.bucket = hostname_matches[1];
}
if (!config.region) {
var s3_domain = hostname_matches[2];
if (s3_domain.slice(0,3) == 's3-' &&
s3_domain.length >= 3) {
// it appears the region is explicit in the url
config.region = s3_domain.replace('s3-','');
}
}
};
+308
View File
@@ -0,0 +1,308 @@
"use strict";
module.exports = exports;
var path = require('path');
var semver = require('semver');
var url = require('url');
var abi_crosswalk;
// This is used for unit testing to provide a fake
// ABI crosswalk that emulates one that is not updated
// for the current version
if (process.env.NODE_PRE_GYP_ABI_CROSSWALK) {
abi_crosswalk = require(process.env.NODE_PRE_GYP_ABI_CROSSWALK);
} else {
abi_crosswalk = require('./abi_crosswalk.json');
}
function get_electron_abi(runtime, target_version) {
if (!runtime) {
throw new Error("get_electron_abi requires valid runtime arg");
}
if (typeof target_version === 'undefined') {
// erroneous CLI call
throw new Error("Empty target version is not supported if electron is the target.");
}
// Electron guarantees that patch version update won't break native modules.
var sem_ver = semver.parse(target_version);
return runtime + '-v' + sem_ver.major + '.' + sem_ver.minor;
}
module.exports.get_electron_abi = get_electron_abi;
function get_node_webkit_abi(runtime, target_version) {
if (!runtime) {
throw new Error("get_node_webkit_abi requires valid runtime arg");
}
if (typeof target_version === 'undefined') {
// erroneous CLI call
throw new Error("Empty target version is not supported if node-webkit is the target.");
}
return runtime + '-v' + target_version;
}
module.exports.get_node_webkit_abi = get_node_webkit_abi;
function get_node_abi(runtime, versions) {
if (!runtime) {
throw new Error("get_node_abi requires valid runtime arg");
}
if (!versions) {
throw new Error("get_node_abi requires valid process.versions object");
}
var sem_ver = semver.parse(versions.node);
if (sem_ver.major === 0 && sem_ver.minor % 2) { // odd series
// https://github.com/mapbox/node-pre-gyp/issues/124
return runtime+'-v'+versions.node;
} else {
// process.versions.modules added in >= v0.10.4 and v0.11.7
// https://github.com/joyent/node/commit/ccabd4a6fa8a6eb79d29bc3bbe9fe2b6531c2d8e
return versions.modules ? runtime+'-v' + (+versions.modules) :
'v8-' + versions.v8.split('.').slice(0,2).join('.');
}
}
module.exports.get_node_abi = get_node_abi;
function get_runtime_abi(runtime, target_version) {
if (!runtime) {
throw new Error("get_runtime_abi requires valid runtime arg");
}
if (runtime === 'node-webkit') {
return get_node_webkit_abi(runtime, target_version || process.versions['node-webkit']);
} else if (runtime === 'electron') {
return get_electron_abi(runtime, target_version || process.versions.electron);
} else {
if (runtime != 'node') {
throw new Error("Unknown Runtime: '" + runtime + "'");
}
if (!target_version) {
return get_node_abi(runtime,process.versions);
} else {
var cross_obj;
// abi_crosswalk generated with ./scripts/abi_crosswalk.js
if (abi_crosswalk[target_version]) {
cross_obj = abi_crosswalk[target_version];
} else {
var target_parts = target_version.split('.').map(function(i) { return +i; });
if (target_parts.length != 3) { // parse failed
throw new Error("Unknown target version: " + target_version);
}
/*
The below code tries to infer the last known ABI compatible version
that we have recorded in the abi_crosswalk.json when an exact match
is not possible. The reasons for this to exist are complicated:
- We support passing --target to be able to allow developers to package binaries for versions of node
that are not the same one as they are running. This might also be used in combination with the
--target_arch or --target_platform flags to also package binaries for alternative platforms
- When --target is passed we can't therefore determine the ABI (process.versions.modules) from the node
version that is running in memory
- So, therefore node-pre-gyp keeps an "ABI crosswalk" (lib/util/abi_crosswalk.json) to be able to look
this info up for all versions
- But we cannot easily predict what the future ABI will be for released versions
- And node-pre-gyp needs to be a `bundledDependency` in apps that depend on it in order to work correctly
by being fully available at install time.
- So, the speed of node releases and the bundled nature of node-pre-gyp mean that a new node-pre-gyp release
need to happen for every node.js/io.js/node-webkit/nw.js/atom-shell/etc release that might come online if
you want the `--target` flag to keep working for the latest version
- Which is impractical ^^
- Hence the below code guesses about future ABI to make the need to update node-pre-gyp less demanding.
In practice then you can have a dependency of your app like `node-sqlite3` that bundles a `node-pre-gyp` that
only knows about node v0.10.33 in the `abi_crosswalk.json` but target node v0.10.34 (which is assumed to be
ABI compatible with v0.10.33).
TODO: use semver module instead of custom version parsing
*/
var major = target_parts[0];
var minor = target_parts[1];
var patch = target_parts[2];
// io.js: yeah if node.js ever releases 1.x this will break
// but that is unlikely to happen: https://github.com/iojs/io.js/pull/253#issuecomment-69432616
if (major === 1) {
// look for last release that is the same major version
// e.g. we assume io.js 1.x is ABI compatible with >= 1.0.0
while (true) {
if (minor > 0) --minor;
if (patch > 0) --patch;
var new_iojs_target = '' + major + '.' + minor + '.' + patch;
if (abi_crosswalk[new_iojs_target]) {
cross_obj = abi_crosswalk[new_iojs_target];
console.log('Warning: node-pre-gyp could not find exact match for ' + target_version);
console.log('Warning: but node-pre-gyp successfully choose ' + new_iojs_target + ' as ABI compatible target');
break;
}
if (minor === 0 && patch === 0) {
break;
}
}
} else if (major === 0) { // node.js
if (target_parts[1] % 2 === 0) { // for stable/even node.js series
// look for the last release that is the same minor release
// e.g. we assume node 0.10.x is ABI compatible with >= 0.10.0
while (--patch > 0) {
var new_node_target = '' + major + '.' + minor + '.' + patch;
if (abi_crosswalk[new_node_target]) {
cross_obj = abi_crosswalk[new_node_target];
console.log('Warning: node-pre-gyp could not find exact match for ' + target_version);
console.log('Warning: but node-pre-gyp successfully choose ' + new_node_target + ' as ABI compatible target');
break;
}
}
}
}
}
if (!cross_obj) {
throw new Error("Unsupported target version: " + target_version);
}
// emulate process.versions
var versions_obj = {
node: target_version,
v8: cross_obj.v8+'.0',
// abi_crosswalk uses 1 for node versions lacking process.versions.modules
// process.versions.modules added in >= v0.10.4 and v0.11.7
modules: cross_obj.node_abi > 1 ? cross_obj.node_abi : undefined
};
return get_node_abi(runtime, versions_obj);
}
}
}
module.exports.get_runtime_abi = get_runtime_abi;
var required_parameters = [
'module_name',
'module_path',
'host'
];
function validate_config(package_json) {
var msg = package_json.name + ' package.json is not node-pre-gyp ready:\n';
var missing = [];
if (!package_json.main) {
missing.push('main');
}
if (!package_json.version) {
missing.push('version');
}
if (!package_json.name) {
missing.push('name');
}
if (!package_json.binary) {
missing.push('binary');
}
var o = package_json.binary;
required_parameters.forEach(function(p) {
if (missing.indexOf('binary') > -1) {
missing.pop('binary');
}
if (!o || o[p] === undefined) {
missing.push('binary.' + p);
}
});
if (missing.length >= 1) {
throw new Error(msg+"package.json must declare these properties: \n" + missing.join('\n'));
}
if (o) {
// enforce https over http
var protocol = url.parse(o.host).protocol;
if (protocol === 'http:') {
throw new Error("'host' protocol ("+protocol+") is invalid - only 'https:' is accepted");
}
}
}
module.exports.validate_config = validate_config;
function eval_template(template,opts) {
Object.keys(opts).forEach(function(key) {
var pattern = '{'+key+'}';
while (template.indexOf(pattern) > -1) {
template = template.replace(pattern,opts[key]);
}
});
return template;
}
// url.resolve needs single trailing slash
// to behave correctly, otherwise a double slash
// may end up in the url which breaks requests
// and a lacking slash may not lead to proper joining
function fix_slashes(pathname) {
if (pathname.slice(-1) != '/') {
return pathname + '/';
}
return pathname;
}
// remove double slashes
// note: path.normalize will not work because
// it will convert forward to back slashes
function drop_double_slashes(pathname) {
return pathname.replace(/\/\//g,'/');
}
function get_process_runtime(versions) {
var runtime = 'node';
if (versions['node-webkit']) {
runtime = 'node-webkit';
} else if (versions.electron) {
runtime = 'electron';
}
return runtime;
}
module.exports.get_process_runtime = get_process_runtime;
var default_package_name = '{module_name}-v{version}-{node_abi}-{platform}-{arch}.tar.gz';
var default_remote_path = '';
module.exports.evaluate = function(package_json,options) {
options = options || {};
validate_config(package_json);
var v = package_json.version;
var module_version = semver.parse(v);
var runtime = options.runtime || get_process_runtime(process.versions);
var opts = {
name: package_json.name,
configuration: Boolean(options.debug) ? 'Debug' : 'Release',
debug: options.debug,
module_name: package_json.binary.module_name,
version: module_version.version,
prerelease: module_version.prerelease.length ? module_version.prerelease.join('.') : '',
build: module_version.build.length ? module_version.build.join('.') : '',
major: module_version.major,
minor: module_version.minor,
patch: module_version.patch,
runtime: runtime,
node_abi: get_runtime_abi(runtime,options.target),
target: options.target || '',
platform: options.target_platform || process.platform,
target_platform: options.target_platform || process.platform,
arch: options.target_arch || process.arch,
target_arch: options.target_arch || process.arch,
module_main: package_json.main,
toolset : options.toolset || '' // address https://github.com/mapbox/node-pre-gyp/issues/119
};
// support host mirror with npm config `--{module_name}_binary_host_mirror`
// e.g.: https://github.com/node-inspector/v8-profiler/blob/master/package.json#L25
// > npm install v8-profiler --profiler_binary_host_mirror=https://npm.taobao.org/mirrors/node-inspector/
var host = process.env['npm_config_' + opts.module_name + '_binary_host_mirror'] || package_json.binary.host;
opts.host = fix_slashes(eval_template(host,opts));
opts.module_path = eval_template(package_json.binary.module_path,opts);
// now we resolve the module_path to ensure it is absolute so that binding.gyp variables work predictably
if (options.module_root) {
// resolve relative to known module root: works for pre-binding require
opts.module_path = path.join(options.module_root,opts.module_path);
} else {
// resolve relative to current working directory: works for node-pre-gyp commands
opts.module_path = path.resolve(opts.module_path);
}
opts.module = path.join(opts.module_path,opts.module_name + '.node');
opts.remote_path = package_json.binary.remote_path ? drop_double_slashes(fix_slashes(eval_template(package_json.binary.remote_path,opts))) : default_remote_path;
var package_name = package_json.binary.package_name ? package_json.binary.package_name : default_package_name;
opts.package_name = eval_template(package_name,opts);
opts.staged_tarball = path.join('build/stage',opts.remote_path,opts.package_name);
opts.hosted_path = url.resolve(opts.host,opts.remote_path);
opts.hosted_tarball = url.resolve(opts.hosted_path,opts.package_name);
return opts;
};
+133
View File
@@ -0,0 +1,133 @@
{
"_args": [
[
{
"raw": "node-pre-gyp@0.6.36",
"scope": null,
"escapedName": "node-pre-gyp",
"name": "node-pre-gyp",
"rawSpec": "0.6.36",
"spec": "0.6.36",
"type": "version"
},
"/Users/linnemann/Documents/dev/nodejs/microservices_boilerplate/service001/node_modules/bcrypt"
]
],
"_from": "node-pre-gyp@0.6.36",
"_id": "node-pre-gyp@0.6.36",
"_inCache": true,
"_location": "/node-pre-gyp",
"_nodeVersion": "6.10.3",
"_npmOperationalInternal": {
"host": "s3://npm-registry-packages",
"tmp": "tmp/node-pre-gyp-0.6.36.tgz_1496189493690_0.5106917407829314"
},
"_npmUser": {
"name": "springmeyer",
"email": "dane@mapbox.com"
},
"_npmVersion": "3.10.10",
"_phantomChildren": {},
"_requested": {
"raw": "node-pre-gyp@0.6.36",
"scope": null,
"escapedName": "node-pre-gyp",
"name": "node-pre-gyp",
"rawSpec": "0.6.36",
"spec": "0.6.36",
"type": "version"
},
"_requiredBy": [
"/bcrypt"
],
"_resolved": "https://registry.npmjs.org/node-pre-gyp/-/node-pre-gyp-0.6.36.tgz",
"_shasum": "db604112cb74e0d477554e9b505b17abddfab786",
"_shrinkwrap": null,
"_spec": "node-pre-gyp@0.6.36",
"_where": "/Users/linnemann/Documents/dev/nodejs/microservices_boilerplate/service001/node_modules/bcrypt",
"author": {
"name": "Dane Springmeyer",
"email": "dane@mapbox.com"
},
"bin": {
"node-pre-gyp": "./bin/node-pre-gyp"
},
"bugs": {
"url": "https://github.com/mapbox/node-pre-gyp/issues"
},
"dependencies": {
"mkdirp": "^0.5.1",
"nopt": "^4.0.1",
"npmlog": "^4.0.2",
"rc": "^1.1.7",
"request": "^2.81.0",
"rimraf": "^2.6.1",
"semver": "^5.3.0",
"tar": "^2.2.1",
"tar-pack": "^3.4.0"
},
"description": "Node.js native addon binary install tool",
"devDependencies": {
"aws-sdk": "^2.28.0",
"jshint": "^2.9.4",
"mocha": "^3.2.0",
"retire": "^1.2.12"
},
"directories": {},
"dist": {
"shasum": "db604112cb74e0d477554e9b505b17abddfab786",
"tarball": "https://registry.npmjs.org/node-pre-gyp/-/node-pre-gyp-0.6.36.tgz"
},
"gitHead": "964a6854e111ec4ea2287a8bf6f012acca755f6a",
"homepage": "https://github.com/mapbox/node-pre-gyp#readme",
"jshintConfig": {
"node": true,
"globalstrict": true,
"undef": true,
"unused": true,
"noarg": true,
"mocha": true
},
"keywords": [
"native",
"addon",
"module",
"c",
"c++",
"bindings",
"binary"
],
"license": "BSD-3-Clause",
"main": "./lib/node-pre-gyp.js",
"maintainers": [
{
"name": "bergwerkgis",
"email": "wb@bergwerk-gis.at"
},
{
"name": "kkaefer",
"email": "mail@kkaefer.com"
},
{
"name": "yhahn",
"email": "young@mapbox.com"
},
{
"name": "springmeyer",
"email": "dane@mapbox.com"
}
],
"name": "node-pre-gyp",
"optionalDependencies": {},
"readme": "ERROR: No README data found!",
"repository": {
"type": "git",
"url": "git://github.com/mapbox/node-pre-gyp.git"
},
"scripts": {
"pretest": "retire -n && jshint test/build.test.js test/s3_setup.test.js test/versioning.test.js",
"test": "jshint lib lib/util scripts bin/node-pre-gyp && mocha -R spec --timeout 500000",
"update-crosswalk": "node scripts/abi_crosswalk.js"
},
"version": "0.6.36"
}
+12
View File
@@ -0,0 +1,12 @@
diff --git a/lib/util/versioning.js b/lib/util/versioning.js
index 8ebbe3a..13e1e0b 100644
--- a/lib/util/versioning.js
+++ b/lib/util/versioning.js
@@ -283,6 +283,7 @@ module.exports.evaluate = function(package_json,options) {
module_main: package_json.main,
toolset : options.toolset || '' // address https://github.com/mapbox/node-pre-gyp/issues/119
};
+ opts.module_name = eval_template(opts.module_name,opts);
// support host mirror with npm config `--{module_name}_binary_host_mirror`
// e.g.: https://github.com/node-inspector/v8-profiler/blob/master/package.json#L25
// > npm install v8-profiler --profiler_binary_host_mirror=https://npm.taobao.org/mirrors/node-inspector/