Update Ruby gem with arm64 binaries (Apple M1 support)

The Ruby gem is shipped with compiled extensions (via https://github.com/rake-compiler/rake-compiler), but it is missing arm64 binaries.

This results in errors like the following when installing the gem and requiring:

$ gem install google-protobuf

$ ruby -e 'require "google/protobuf"'
Traceback (most recent call last):
	2: from -e:1:in `<main>'
	1: from <snip>/rubygems/core_ext/kernel_require.rb:92:in `require'
<snip>/rubygems/core_ext/kernel_require.rb:92:in `require': cannot load such file -- google/protobuf (LoadError)
	6: from -e:1:in `<main>'
	5: from <snip>/rubygems/core_ext/kernel_require.rb:156:in `require'
	4: from <snip>/rubygems/core_ext/kernel_require.rb:168:in `rescue in require'
	3: from <snip>/rubygems/core_ext/kernel_require.rb:168:in `require'
	2: from <snip>/gems/google-protobuf-3.14.0-universal-darwin/lib/google/protobuf.rb:49:in `<top (required)>'
	1: from <snip>/rubygems/core_ext/kernel_require.rb:92:in `require'
<snip>/rubygems/core_ext/kernel_require.rb:92:in `require': dlopen(<snip>/gems/google-protobuf-3.14.0-universal-darwin/lib/google/2.7/protobuf_c.bundle, 9): no suitable image found.  Did find: (LoadError)
	<snip>/gems/google-protobuf-3.14.0-universal-darwin/lib/google/2.7/protobuf_c.bundle: mach-o, but wrong architecture
	<snip>/gems/google-protobuf-3.14.0-universal-darwin/lib/google/2.7/protobuf_c.bundle: mach-o, but wrong architecture - <snip>/gems/google-protobuf-3.14.0-universal-darwin/lib/google/2.7/protobuf_c.bundle
	7: from -e:1:in `<main>'
	6: from <snip>/rubygems/core_ext/kernel_require.rb:156:in `require'
	5: from <snip>/rubygems/core_ext/kernel_require.rb:168:in `rescue in require'
	4: from <snip>/rubygems/core_ext/kernel_require.rb:168:in `require'
	3: from <snip>/gems/google-protobuf-3.14.0-universal-darwin/lib/google/protobuf.rb:48:in `<top (required)>'
	2: from <snip>/gems/google-protobuf-3.14.0-universal-darwin/lib/google/protobuf.rb:51:in `rescue in <top (required)>'
	1: from <snip>/rubygems/core_ext/kernel_require.rb:92:in `require'
<snip>/rubygems/core_ext/kernel_require.rb:92:in `require': cannot load such file -- google/protobuf_c (LoadError)

A workaround is to force compilation of native extensions on install which works perfectly:

$ gem install google-protobuf --platform=ruby

or globally via this bundler setting:

$ bundle config set force_ruby_platform true

Related to #8062

Confirmed that

$ bundle config set force_ruby_platform true

in combination with grpc v1.35.0 upgrade solved this issue for me.

This still does not work for me. I have v. 3.15.3 installed and it throws this:

Traceback (most recent call last):
       16: from bootsnap (1.7.0) lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:23:in `require'
       15: from google-protobuf-3.15.3-universal (darwin) lib/google/protobuf/any_pb.rb:4:in `<main>'
       14: from zeitwerk (2.4.2) lib/zeitwerk/kernel.rb:34:in `require'
       13: from bootsnap (1.7.0) lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:31:in `require'
       12: from bootsnap (1.7.0) lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:22:in `require_with_bootsnap_lfi'
       11: from bootsnap (1.7.0) lib/bootsnap/load_path_cache/loaded_features_index.rb:92:in `register'
       10: from bootsnap (1.7.0) lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:23:in `block in require_with_bootsnap_lfi'
        9: from bootsnap (1.7.0) lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:23:in `require'
        8: from google-protobuf-3.15.3-universal (darwin) lib/google/protobuf.rb:48:in `<main>'
        7: from google-protobuf-3.15.3-universal (darwin) lib/google/protobuf.rb:51:in `rescue in <main>'
        6: from zeitwerk (2.4.2) lib/zeitwerk/kernel.rb:34:in `require'
        5: from bootsnap (1.7.0) lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:44:in `require'
        4: from bootsnap (1.7.0) lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:22:in `require_with_bootsnap_lfi'
        3: from bootsnap (1.7.0) lib/bootsnap/load_path_cache/loaded_features_index.rb:89:in `register'
        2: from bootsnap (1.7.0) lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:23:in `block in require_with_bootsnap_lfi'
        1: from bootsnap (1.7.0) lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:23:in `require'
LoadError (cannot load such file -- google/protobuf_c)

If I try to load the gem locally and alter lines inside lib/google/protobuf.rb like so:

  # begin
    require "google/#{RUBY_VERSION.sub(/\.\d+$/, '')}/protobuf_c"
  # rescue LoadError
  #   require 'google/protobuf_c'
  # end

I get:

LoadError (dlopen(/Users/niels-kristian/.rvm/gems/ruby-2.7.2@rails6/gems/google-protobuf-3.15.3-universal-darwin/lib/google/2.7/protobuf_c.bundle, 9): no suitable image found.  Did find:)
	/Users/niels-kristian/.rvm/gems/ruby-2.7.2@rails6/gems/google-protobuf-3.15.3-universal-darwin/lib/google/2.7/protobuf_c.bundle: mach-o, but wrong architecture
	/Users/niels-kristian/.rvm/gems/ruby-2.7.2@rails6/gems/google-protobuf-3.15.3-universal-darwin/lib/google/2.7/protobuf_c.bundle: stat() failed with errno=25 - /Users/niels-kristian/.rvm/gems/ruby-2.7.2@rails6/gems/google-protobuf-3.15.3-universal-darwin/lib/google/2.7/protobuf_c.bundle

Any clue why this happens?

Okay I tried updating to v 3.15.4 then it worked. 1 hour later it suddenly stopped working again 🤯 Something is really fishy...

LoadError (dlopen(/Users/niels-kristian/.rvm/gems/ruby-2.7.2@rails6/gems/google-protobuf-3.15.4-universal-darwin/lib/google/2.7/protobuf_c.bundle, 9): no suitable image found.  Did find:)
	/Users/niels-kristian/.rvm/gems/ruby-2.7.2@rails6/gems/google-protobuf-3.15.4-universal-darwin/lib/google/2.7/protobuf_c.bundle: mach-o, but wrong architecture
	/Users/niels-kristian/.rvm/gems/ruby-2.7.2@rails6/gems/google-protobuf-3.15.4-universal-darwin/lib/google/2.7/protobuf_c.bundle: mach-o, but wrong architecture - /Users/niels-kristian/.rvm/gems/ruby-2.7.2@rails6/gems/google-protobuf-3.15.4-universal-darwin/lib/google/2.7/protobuf_c.bundle

I just tried to checkout the ruby gem and see if it could build, it cannot: #8372

solved on my new M1 Mac, try the following steps:

gem uninstall google-protobuf, and remove all google-protobuf
reinstall with gem install google-protobuf -v 3.15.8 --platform=ruby or any version you want

seems bundle config force ruby platform not working, we have to reinstall google-protobuf platform: ruby from scratch

None of these solutions worked for me on 3.17.3 on a 13" M1 MBP. I'm running the latest Monterey public beta (beta 5 I believe) so maybe that's the issue?

Installation went fine but when I go to require it I get a "No such file" error when looking for the arm64e bundle. Here's the full error:

.bundle/bundle/ruby/2.7.0/gems/activesupport-6.1.4.1/lib/active_support/dependencies.rb:332:in `require': dlopen(.bundle/bundle/ruby/2.7.0/gems/google-protobuf-3.17.3-universal-darwin/lib/google/2.7/protobuf_c.bundle, 0x0009): tried: '.bundle/bundle/ruby/2.7.0/gems/google-protobuf-3.17.3-universal-darwin/lib/google/2.7/protobuf_c.bundle' (mach-o file, but is an incompatible architecture (have 'x86_64', need 'arm64e')), '/usr/local/lib/protobuf_c.bundle' (no such file), '/usr/lib/protobuf_c.bundle' (no such file) - .bundle/bundle/ruby/2.7.0/gems/google-protobuf-3.17.3-universal-darwin/lib/google/2.7/protobuf_c.bundle (LoadError)

I tried on Ruby 2.6.8p205 and 2.7.4p191. I tried recompiling Ruby several times using Xcode Command Line tools and the full version of Xcode (Mac App Store).

Is there anyway to force bundler to use the Rubygems version installed with --platform ruby ?

Whenever I run bundle I see it's using the universal-darwin version.
I've tried doing platform: :ruby to the line in the Gemfile but that doesn't work.

If you want to prevent bundler from installing these gems, add the following to you Gemfile:

# HACK(bouk): Overwrite Bundler's platform matcher to ignore universal CPU
# The protobuf and gRPC 'universal' macOS gems break on M1
module Bundler::MatchPlatform
  def match_platform(p)
    return false if ::Gem::Platform === platform && platform.cpu == "universal"
    Bundler::MatchPlatform.platforms_match?(platform, p)
  end
end

This will overwrite bundler's platform matcher to ignore any universal gems

This was closed by #8232 (and released in 3.19.0).

I'm not sure if this is actually fixed. I'm still getting the same problem (protobuf_c.bundle: mach-o, but wrong architecture) after upgrading to 3.19.0.

I also don't see an arm64-darwin build for 3.19.0:

I think some more work is required on the protobuf team's part before we can start publishing Mac arm64 binaries. The build is happening on an x86 machine and we probably need to tweak some things to get cross-compilation working.

+1 for this

I've tried all of the suggestions above, but still doesn't work for me:

I get this error:

/Users/dangerrg/.rvm/gems/ruby-3.0.3/gems/bootsnap-1.9.3/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:23:in `
require': dlopen(/Users/dangerrg/.rvm/gems/ruby-3.0.3/gems/digest-3.1.0/lib/digest.bundle, 0x0009): tried: '/Users/dange
rrg/.rvm/gems/ruby-3.0.3/gems/digest-3.1.0/lib/digest.bundle' (mach-o file, but is an incompatible architecture (have 'x
86_64', need 'arm64e')), '/usr/local/lib/digest.bundle' (no such file), '/usr/lib/digest.bundle' (no such file) - /Users
/dangerrg/.rvm/gems/ruby-3.0.3/gems/digest-3.1.0/lib/digest.bundle (LoadError)

I'm running on Mac M1:

About your application's environment
Rails version             7.0.0
Ruby version              ruby 3.0.3p157 (2021-11-24 revision 3fb7d2cadc) [arm64-darwin21]
RubyGems version          3.2.32
Rack version              2.2.3
Middleware                ActionDispatch::HostAuthorization, Rack::Sendfile, ActionDispatch::Static, ActionDispatch::Exe
cutor, ActionDispatch::ServerTiming, ActiveSupport::Cache::Strategy::LocalCache::Middleware, Rack::Runtime, Rack::Method
Override, ActionDispatch::RequestId, ActionDispatch::RemoteIp, Sprockets::Rails::QuietAssets, Rails::Rack::Logger, Actio
nDispatch::ShowExceptions, WebConsole::Middleware, ActionDispatch::DebugExceptions, ActionDispatch::ActionableExceptions
, ActionDispatch::Reloader, ActionDispatch::Callbacks, ActiveRecord::Migration::CheckPending, ActionDispatch::Cookies, A
ctionDispatch::Session::CookieStore, ActionDispatch::Flash, ActionDispatch::ContentSecurityPolicy::Middleware, ActionDis
patch::PermissionsPolicy::Middleware, Rack::Head, Rack::ConditionalGet, Rack::ETag, Rack::TempfileReaper
Application root          /Users/dangerrg/Documents/rails_7_projects/rails_7_blog
Environment               development
Database adapter          sqlite3
Database schema version   20220105220427

Updates on this issue? It appears to be ongoing and I'm only able to bundle with bundle config set force_ruby_platform true

@deannagarcia

Similar to #8682, closing this because in version 3.22 protobuf is planning to experiment with releasing only a source gem.

If this experiment is unsuccessful and we revert to building binary gems for some platforms, we'll evaluate whether it makes sense to add arm64 darwin at that time.

CC: @deannagarcia @haberman

I am running this on M2, facing the same issue

``require': dlopen(/Library/Ruby/Gems/2.6.0/gems/google-protobuf-3.21.12-x86_64-darwin/lib/google/2.6/protobuf_c.bundle, 0x0009): tried: '/Library/Ruby/Gems/2.6.0/gems/google-protobuf-3.21.12-x86_64-darwin/lib/google/2.6/protobuf_c.bundle' (mach-o file, but is an incompatible architecture (have 'x86_64', need 'arm64')), '/System/Volumes/Preboot/Cryptexes/OS/Library/Ruby/Gems/2.6.0/gems/google-protobuf-3.21.12-x86_64-darwin/lib/google/2.6/protobuf_c.bundle' (no such file), '/Library/Ruby/Gems/2.6.0/gems/google-protobuf-3.21.12-x86_64-darwin/lib/google/2.6/protobuf_c.bundle' (mach-o file, but is an incompatible architecture (have 'x86_64', need 'arm64')) - /Library/Ruby/Gems/2.6.0/gems/google-protobuf-3.21.12-x86_64-darwin/lib/google/2.6/protobuf_c.bundle (LoadError)`

dont know what to do?

I am running this on M2, facing the same issue

``require': dlopen(/Library/Ruby/Gems/2.6.0/gems/google-protobuf-3.21.12-x86_64-darwin/lib/google/2.6/protobuf_c.bundle, 0x0009): tried: '/Library/Ruby/Gems/2.6.0/gems/google-protobuf-3.21.12-x86_64-darwin/lib/google/2.6/protobuf_c.bundle' (mach-o file, but is an incompatible architecture (have 'x86_64', need 'arm64')), '/System/Volumes/Preboot/Cryptexes/OS/Library/Ruby/Gems/2.6.0/gems/google-protobuf-3.21.12-x86_64-darwin/lib/google/2.6/protobuf_c.bundle' (no such file), '/Library/Ruby/Gems/2.6.0/gems/google-protobuf-3.21.12-x86_64-darwin/lib/google/2.6/protobuf_c.bundle' (mach-o file, but is an incompatible architecture (have 'x86_64', need 'arm64')) - /Library/Ruby/Gems/2.6.0/gems/google-protobuf-3.21.12-x86_64-darwin/lib/google/2.6/protobuf_c.bundle (LoadError)`

dont know what to do?

I resolved the issue by ditching homebrew and following the ruby installation docs provided on Jekyll site

solved on my new M1 Mac, try the following steps:

gem uninstall google-protobuf, and remove all google-protobuf

reinstall with gem install google-protobuf -v 3.15.8 --platform=ruby or any version you want

seems bundle config force ruby platform not working, we have to reinstall google-protobuf platform: ruby from scratch

This really helps after searching for a lot methods.

Protobuf is releasing with arm64-darwin binary gems. For example, here is the latest one: https://rubygems.org/gems/google-protobuf/versions/3.24.2-arm64-darwin

However, the current gems require Ruby >=2.7, and I believe macOS is still shipping with Ruby 2.6.

So to use the current version of the protobuf package, you would need to install a separate Ruby from Homebrew (or from source).

Navigation Menu

Repository navigation

Update Ruby gem with arm64 binaries (Apple M1 support) #8199

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Update Ruby gem with arm64 binaries (Apple M1 support) #8199

Description

Activity

LucaProvencal commented on Feb 4, 2021

NielsKSchjoedt commented on Mar 4, 2021

NielsKSchjoedt commented on Mar 4, 2021

teddy1004 commented on May 17, 2021

ericboehs commented on Aug 30, 2021

Overload119 commented on Sep 8, 2021

bouk commented on Oct 13, 2021

mloughran commented on Oct 25, 2021

ngan commented on Oct 25, 2021

acozzette commented on Oct 28, 2021

jnarowski commented on Dec 15, 2021

dangerrg commented on Jan 7, 2022

nickmealey commented on Jun 21, 2022

JasonLunn commented on Sep 1, 2022

njmulsqb commented on Jan 3, 2023

njmulsqb commented on Jan 16, 2023

Hazel233 commented on Aug 25, 2023

haberman commented on Aug 28, 2023

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions