nodegit in alpine docker image fails with __rawmemchr: symbol not found

~ $ node -v
v5.10.1
~ $ npm -v
3.8.3

npm install nodegit;npm ls

> nodegit@0.13.0 install /home/user/node_modules/nodegit
> node lifecycleScripts/install

[nodegit] Fetching binary from S3.
[nodegit] Completed installation successfully.

> nodegit@0.13.0 postinstall /home/user/node_modules/nodegit

/home/user
`-- nodegit@0.13.0 
  +-- fs-extra@0.26.7 
  | +-- graceful-fs@4.1.4 
  | +-- jsonfile@2.3.1 
  | +-- klaw@1.2.0 
  | +-- path-is-absolute@1.0.0 
  | `-- rimraf@2.5.2 
  |   `-- glob@7.0.3 
  |     +-- inflight@1.0.5 
  |     | `-- wrappy@1.0.2 
  |     +-- inherits@2.0.1 
  |     +-- minimatch@3.0.0 
  |     | `-- brace-expansion@1.1.4 
  |     |   +-- balanced-match@0.4.1 
  |     |   `-- concat-map@0.0.1 
  |     `-- once@1.3.3 
  +-- node-pre-gyp@0.6.27 
  | +-- mkdirp@0.5.1 
  | | `-- minimist@0.0.8 
  | +-- nopt@3.0.6 
  | | `-- abbrev@1.0.7 
  | +-- npmlog@2.0.3 
  | | +-- ansi@0.3.1 
  | | +-- are-we-there-yet@1.1.2 
  | | | +-- delegates@1.0.0 
  | | | `-- readable-stream@2.1.2 
  | | `-- gauge@1.2.7 
  | |   +-- has-unicode@2.0.0 
  | |   +-- lodash.pad@4.3.0 
  | |   | +-- lodash._baseslice@4.0.0 
  | |   | `-- lodash.tostring@4.1.2 
  | |   +-- lodash.padend@4.4.0 
  | |   `-- lodash.padstart@4.4.0 
  | +-- rc@1.1.6 
  | | +-- deep-extend@0.4.1 
  | | +-- ini@1.3.4 
  | | +-- minimist@1.2.0 
  | | `-- strip-json-comments@1.0.4 
  | +-- request@2.72.0 
  | | +-- aws-sign2@0.6.0 
  | | +-- aws4@1.3.2 
  | | | `-- lru-cache@4.0.1 
  | | |   +-- pseudomap@1.0.2 
  | | |   `-- yallist@2.0.0 
  | | +-- bl@1.1.2 
  | | | `-- readable-stream@2.0.6 
  | | +-- caseless@0.11.0 
  | | +-- combined-stream@1.0.5 
  | | | `-- delayed-stream@1.0.0 
  | | +-- extend@3.0.0 
  | | +-- forever-agent@0.6.1 
  | | +-- form-data@1.0.0-rc4 
  | | | `-- async@1.5.2 
  | | +-- har-validator@2.0.6 
  | | | +-- chalk@1.1.3 
  | | | | +-- ansi-styles@2.2.1 
  | | | | +-- escape-string-regexp@1.0.5 
  | | | | +-- has-ansi@2.0.0 
  | | | | | `-- ansi-regex@2.0.0 
  | | | | +-- strip-ansi@3.0.1 
  | | | | `-- supports-color@2.0.0 
  | | | +-- commander@2.9.0 
  | | | | `-- graceful-readlink@1.0.1 
  | | | +-- is-my-json-valid@2.13.1 
  | | | | +-- generate-function@2.0.0 
  | | | | +-- generate-object-property@1.2.0 
  | | | | | `-- is-property@1.0.2 
  | | | | +-- jsonpointer@2.0.0 
  | | | | `-- xtend@4.0.1 
  | | | `-- pinkie-promise@2.0.1 
  | | |   `-- pinkie@2.0.4 
  | | +-- hawk@3.1.3 
  | | | +-- boom@2.10.1 
  | | | +-- cryptiles@2.0.5 
  | | | +-- hoek@2.16.3 
  | | | `-- sntp@1.0.9 
  | | +-- http-signature@1.1.1 
  | | | +-- assert-plus@0.2.0 
  | | | +-- jsprim@1.2.2 
  | | | | +-- extsprintf@1.0.2 
  | | | | +-- json-schema@0.2.2 
  | | | | `-- verror@1.3.6 
  | | | `-- sshpk@1.8.3 
  | | |   +-- asn1@0.2.3 
  | | |   +-- assert-plus@1.0.0 
  | | |   +-- dashdash@1.13.1 
  | | |   | `-- assert-plus@1.0.0 
  | | |   +-- ecc-jsbn@0.1.1 
  | | |   +-- getpass@0.1.6 
  | | |   | `-- assert-plus@1.0.0 
  | | |   +-- jodid25519@1.0.2 
  | | |   +-- jsbn@0.1.0 
  | | |   `-- tweetnacl@0.13.3 
  | | +-- is-typedarray@1.0.0 
  | | +-- isstream@0.1.2 
  | | +-- json-stringify-safe@5.0.1 
  | | +-- mime-types@2.1.11 
  | | | `-- mime-db@1.23.0 
  | | +-- node-uuid@1.4.7 
  | | +-- oauth-sign@0.8.2 
  | | +-- qs@6.1.0 
  | | +-- stringstream@0.0.5 
  | | +-- tough-cookie@2.2.2 
  | | `-- tunnel-agent@0.4.3 
  | +-- rimraf@2.5.2 
  | | `-- glob@7.0.3 
  | |   +-- inflight@1.0.4 
  | |   +-- minimatch@3.0.0 
  | |   | `-- brace-expansion@1.1.4 
  | |   |   +-- balanced-match@0.4.1 
  | |   |   `-- concat-map@0.0.1 
  | |   `-- path-is-absolute@1.0.0 
  | +-- semver@5.1.0 
  | +-- tar@2.2.1 
  | | +-- block-stream@0.0.9 
  | | +-- fstream@1.0.8 
  | | | `-- graceful-fs@4.1.3 
  | | `-- inherits@2.0.1 
  | `-- tar-pack@3.1.3 
  |   +-- debug@2.2.0 
  |   | `-- ms@0.7.1 
  |   +-- fstream-ignore@1.0.4 
  |   +-- once@1.3.3 
  |   | `-- wrappy@1.0.1 
  |   +-- readable-stream@2.0.6 
  |   | +-- core-util-is@1.0.2 
  |   | +-- isarray@1.0.0 
  |   | +-- process-nextick-args@1.0.7 
  |   | +-- string_decoder@0.10.31 
  |   | `-- util-deprecate@1.0.2 
  |   `-- uid-number@0.0.6 
  `-- promisify-node@0.3.0 
    `-- nodegit-promise@4.0.0 
      `-- asap@2.0.4 
~ $ node
> require('nodegit')
Error: Error relocating /home/bdb/node_modules/nodegit/build/Release/nodegit.node: __rawmemchr: symbol not found
    at Error (native)
    at Object.Module._extensions..node (module.js:440:18)
    at Module.load (module.js:357:32)
    at Function.Module._load (module.js:314:12)
    at Module.require (module.js:367:17)
    at require (internal/module.js:16:19)
    at Object.<anonymous> (/home/bdb/node_modules/nodegit/dist/nodegit.js:11:12)
    at Module._compile (module.js:413:34)
    at Object.Module._extensions..js (module.js:422:10)
    at Module.load (module.js:357:32)

Looks like a limitation of alpine's musl libc? Java 8 has a similar problem gliderlabs/docker-alpine#11

Maybe compiling from source may work.

Just got the same error on now.sh which I think uses Heroku Alpine.

I don't think NodeGit has ever been successfully containerized. If anyone gets it working please report back on what you did and we'll throw up a guide on the site.

@tbranyen

@tbranyen: do you have any recommendation on container images for running nodejs apps that depends on nodegit?

I don't know anyone who has run NodeGit in a docker image. Would be great if someone could figure it out. We'll add a guide once someone does.

@johnhaley81

At Cucumber Ltd, we run NodeGit in a docker container. Our base image is alpine-node (node 6.1.0, but node 5.11.x works as well).
The pre-built binary is not working because of musl libc limitations indeed. So we use the following command to install node packages:

$ BUILD_ONLY=true npm install

This according to the guide will force a build instead of relying on fetched binaries. With the current version of nodegit (0.13.0), a bug in the install scripts prevents us from building the binary via npm.
It has been fixed here: d8dcc0f, thanks @johnhaley81!
For now we are stuck on version 0.12.2.

Is it possible to release a new version of nodegit that includes this change soon, so we could test nodegit 0.13.x in that environment?

@romaingweb

@romaingweb https://github.com/nodegit/nodegit/releases/tag/v0.13.1

@johnhaley81

@johnhaley81 That was quick! Thanks again!
Just tried it inside a docker image and it's working, kind of ;-). I ran into a weird issue.
At first I thought that was the same problem as before but then I did some logging on the install/build process and it looks like the submodules/getStatus thing in there depends on the fact that the parent app (or at least two levels above the node_modules/nodegit) is a git repository (containing a .git folder). Inside a container obviously you want to git rid of as many things as you can to keep it small so it's not the case.

For now just before running npm install I initialise an empty repo. That does the trick. Is there any particular reason for that git submodule status?

Actually it is a bit trickier than I thought. The getStatus() expects the npm package to be a git repository (which is not the case apparently), making my own in the app just allowed for git commands to run properly. Maybe something changed in the way npm is pulling git submodules? We are using npm@3.9.5 in the image.

@romaingweb

@romaingweb it needs to download the version of libgit2 it's building against. If there isn't a prebuilt available then it needs to go grab libgit2 to compile the binary.

Ok, that I understand. So the only thing that's not happening inside the docker container/image is the initialisation/presence of a git repository for some reason. I tracked that down by catching the error of the exec command:

.catch(function(err) {
  if (err.message.indexOf("Not a git repository")) {
    return Promise.resolve([]);
  } else {
    throw err;
  }
})

npm must be stripping the package of all git related data. Is NodeGit a dependency or a sub-dependency?

It's a direct dependency of our app. Prior to version 0.13.0 it was working though. I can check our old containers for .git folders in node_modules/nodegit but I'm pretty sure it was there. It's the only reason that status command could have worked. I'm digging into the install process to see what might have changed.

Ahh, v0.13 is when libgit2 became a submodule. So what we could do is have a version OID plopped into the root package.json file that we could use as a backup and then do a git clone of that if the submodule stuff doesn't work.

So basically, the fact that alpine comes with some version of libgit2 does not trigger the submodule pull (and indirectly initialising a git repository), which causes the failing attempt to check the status of submodules?

I'm not too sure why it's dying. In my project we have to build NodeGit but it doesn't get the submodule stuff stripped. Or it does but it resolves it correctly. I'll have to look into why it works for me.

Maybe for the same reason it is working for us inside the image when we create a git repository for our app, you can run git commands inside the package as well; Could you try to do BUILD_ONLY=true npm install inside the package but within a parent app that is not a git repository? (by moving the .git to .git_bis temporarily for example)

@romaingweb

@romaingweb can you try going to npm 3.8.9?

I did, it doesn't work either. So npm didn't change this behaviour in 3.9.x.

I cooked up a little scenario to reproduce this problem very easily:

create a new directory and npm init inside it;
run BUILD_ONLY=true npm install --save nodegit@0.13.1
check the output or build directory in the module, nothing has been built.

Like I suspected this getStatus() function only works inside a git repository, it can be the inside the module itself or any ancestor directory. If none is found by git, git submodule status or any other git status/log command will fail with the following message: fatal: Not a git repository (or any of the parent directories): .git

The install script really relies on the app requiring NodeGit to be a git directory to be able to check the submodule status, or that a git directory has been initialised inside the module.

Possible solution: https://gist.github.com/romaingweb/6f5940e68424e3fd7eab07546fe502f1

I've been able to get nodegit to successfully install with the Docker node image, using a Dockerfile like this (using the latest supported node image, and the highest published nodegit version - as the latest published was 0.14.1 I have explicitly reference the highest version, not the latest version):

# Versions of node >= 6.3 are currently not supported by nodegit/node-pre-gyp
FROM node:6.2.0

# Required to not run in the root directory
WORKDIR /src

# Set unsafe perms when run as root, or, alternatively, run as a non-root user
RUN npm init -y && \
    npm config set unsafe-perm true && \
    npm install --production nodegit@0.15.1

Or:

# Versions of node >= 6.3 are currently not supported by nodegit/node-pre-gyp
FROM node:6.2.0

# Required to not run in the root directory
WORKDIR /src

# Set unsafe perms when run as root, or, alternatively, run as a non-root user
RUN npm init -y && \
    npm config set unsafe-perm true && \
    npm install nodegit@0.15.1 && \
    npm prune --production

Do I understand correctly that the Alpine problem is worked around by forcing building from source. The remaining issue is getting the libgit2 submodule recognized when unpacked via npm@3?

I managed to have nodegit installed from binaries on an alpine image. The trick was to use an alpine:3.6+ image. Any attempt to use an alpine image below 3.6 resulted in the OP mentioned error.
If anyone knows what changed in alpine 3.6. please share.
For those that need to use a node version that is built with an alpine image below 3.6, they will have to build their own node-alpine image (see how @ https://github.com/nodejs/docker-node).

The is a new version of musl and a new version of node. If the musl node-gyp has been built with 3.6+ you cannot use it with a lower musl version: you need to recompile

Navigation Menu

Repository navigation

nodegit in alpine docker image fails with __rawmemchr: symbol not found #1039

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

nodegit in alpine docker image fails with __rawmemchr: symbol not found #1039

Description

Activity

saper commented on May 29, 2016

apaleslimghost commented on May 29, 2016

tbranyen commented on May 29, 2016

hc000 commented on Jun 1, 2016

tbranyen commented on Jun 1, 2016

romaingweb commented on Jun 3, 2016

johnhaley81 commented on Jun 3, 2016

romaingweb commented on Jun 6, 2016

romaingweb commented on Jun 6, 2016

johnhaley81 commented on Jun 6, 2016

romaingweb commented on Jun 6, 2016

johnhaley81 commented on Jun 6, 2016

romaingweb commented on Jun 6, 2016

johnhaley81 commented on Jun 6, 2016

romaingweb commented on Jun 6, 2016

johnhaley81 commented on Jun 6, 2016

romaingweb commented on Jun 6, 2016

johnhaley81 commented on Jun 6, 2016

romaingweb commented on Jun 6, 2016

romaingweb commented on Jun 6, 2016

fastbean-au commented on Aug 19, 2016

saper commented on Aug 20, 2016

JimiC commented on Feb 6, 2018

saper commented on Feb 6, 2018

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions