Hi can anyone point me in the direction of what I am doing wrong. I am trying to upload an image to an S3 bucket in EU region, Ireland, using a rails application in development. This is the error I am getting the error
SocketError in ProductsController
getaddrinfo: nodename nor servname provided, or not known
On the following line : TCPSocket.open(conn_address, conn_port, #local_host, #local_port)
I am using the aws-sdk gem and paperclip 4.2.
Here are some code snippets
In:
config/enviroments/development.rb
config.paperclip_defaults = {
:storage => :s3,
:s3_host_name => "s3-eu-west-1.amazonaws.com",
:s3_credentials => {
:bucket => 'S3_BUCKET_NAME',
:access_key_id => 'AWS_ACCESS_KEY_ID',
:secret_access_key => 'AWS_SECRET_ACCESS_KEY'
}
In
config/aws.yml
development:
AWS_ACCESS_KEY_ID: "xxx"
AWS_SECRET_ACCESS_KEY: "xxx"
S3_BUCKET_NAME: "xxx"
s3_host_name: 's3-eu-west-1.amazonaws.com'
In my model
class Product < ActiveRecord::Base
validates :avatar,
attachment_content_type: { content_type: /\Aimage\/.*\Z/ },
attachment_size: { less_than: 5.megabytes }
has_attached_file :avatar, styles: {
thumb: '100x100>',
square: '200x200#',
medium: '300x300>'
}
end
What am I missing? I have looked at every example I could find online and tried to adjust but with no luck.
Thanks
Try this:
#config/enviroments/development.rb
config.paperclip_defaults = {
:storage => :s3,
:s3_host_name => "s3-eu-west-1.amazonaws.com",
:s3_credentials => {
:bucket => ENV["S3_BUCKET_NAME"],
:access_key_id => ENV["AWS_ACCESS_KEY_ID"],
:secret_access_key => ENV["AWS_SECRET_ACCESS_KEY"]
}
Ok, I've solved this by hardcoding the AWS credentials into the development.rb file. So it looks like the problem was the aws.yml file wasn't being loaded, does anyone have any idea why or how to solve this?
Thanks again
Related
I'm working on an Rails 4.1 engine that handles user uploads of photos and videos. I'm using Mongoid-Paperclip to handle the uploads and Paperclip-av-transcoder to encode the videos into several formats. All files are stored at S3. All of that works fine, but as you can expect, encoding the videos can take quite some time, so the next step is to make that happen in the background. I did some googling and found Delayed_Paperclip that seems to do what I need. After that it seemed that Sidekiq was the best option for handling the background processing.
Now the problem is, I can't make all this work together. Running my unit tests I get NoMethodError: undefined method 'process_in_background' so it seems the problem resides on Delayed_Paperclip, although there is no special setup to it.
This is the model firing the problem
module MyEngine
class Video
include Mongoid::Document
include Mongoid::Paperclip
has_mongoid_attached_file :file,
:path => ':hash.:extension',
:hash_secret => "the-secret",
:storage => :s3,
:url => ':s3_domain_url',
:s3_credentials => File.join(Rails.root, 'config', 's3.yml'),
:bucket => "my-bucket-#{Rails.env}",
:styles => {
:mp4 => { :format => 'mp4', :convert_options => { :output => { :vcodec => 'libx264', :acodec => 'copy' } } },
:ogg => { :format => 'ogg', :auto_rotate => true },
:webm => { :format => 'webm', :auto_rotate => true },
:thumb => { :geometry => "250x187#", :format => 'jpg', :time => 10, :auto_rotate => true }
},
:processors => [:transcoder]
validates_attachment :file, :content_type => { :content_type => ["video/x-flv", "video/mp4", "video/ogg", "video/webm", "video/x-ms-wmv", "video/x-msvideo", "video/quicktime", "video/3gpp"] }
process_in_background :file
end
end
I've tried adding require "delayed_paperclip" to the lib/myengine/myengine.rb file but that didn't help.
Regarding Sidekiq, I have added to the test_helper.rb the following:
require 'sidekiq/testing'
Sidekiq::Testing.inline!
Note that I did not forget to run bundle install and Redis is up and running. I'm using Mongoid, not active record.
What I am doing wrong? Has anyone successfully used this setup? Is there another combination of gems that I should try?
Aditional info:
Delayed_paperclip 2.9.1
Mongoid 4.0.2
Mongoid-Paperclip 0.0.9
Paperclip 4.2.1
Paperclip-av-transcoder 0.6.4
Rails 4.1.9
Sidekiq 3.5.0
I've been digging through the code of delayed_paperclip and it is definitely tied to ActiveRecord so not compatible with Mongoid. I gave a try to mongoid_paperclip_queue but that gem hasn't been updated in 4 years and doesn't seem to work with the current versions of rails/mongoid/paperclip as far as I can tell.
I therefore decided the best way to solve my issue would be to override the code of delayed_paperclip that integrates with ActiveRecord and make it instead work with Mongoid.
This is what I ended up doing, and seems to be working fine so far:
lib/myengine.rb
require "mongoid_paperclip"
require "paperclip/av/transcoder"
require "delayed_paperclip"
require "myengine/engine"
module Myengine
end
DelayedPaperclip::Railtie.class_eval do
initializer 'delayed_paperclip.insert_into_mongoid' do |app|
ActiveSupport.on_load :mongoid do
DelayedPaperclip::Railtie.insert
end
if app.config.respond_to?(:delayed_paperclip_defaults)
DelayedPaperclip.options.merge!(app.config.delayed_paperclip_defaults)
end
end
# Attachment and URL Generator extends Paperclip
def self.insert
Paperclip::Attachment.send(:include, DelayedPaperclip::Attachment)
Paperclip::UrlGenerator.send(:include, DelayedPaperclip::UrlGenerator)
end
end
DelayedPaperclip::InstanceMethods.class_eval do
def enqueue_post_processing_for name
DelayedPaperclip.enqueue(self.class.name, read_attribute(:id).to_s, name.to_sym)
end
end
Then all you need is to include the delayed_paperclip glue to the model:
module Myengine
class Video
include Mongoid::Document
include Mongoid::Paperclip
include DelayedPaperclip::Glue # <---- Include this
has_mongoid_attached_file :file,
:path => ':hash.:extension',
:hash_secret => "the-secret",
:storage => :s3,
:url => ':s3_domain_url',
:s3_credentials => File.join(Rails.root, 'config', 's3.yml'),
:bucket => "my-bucket-#{Rails.env}",
:styles => {
:mp4 => { :format => 'mp4', :convert_options => { :output => { :vcodec => 'libx264', :acodec => 'copy' } } },
:ogg => { :format => 'ogg', :auto_rotate => true },
:webm => { :format => 'webm', :auto_rotate => true },
:thumb => { :geometry => "250x187#", :format => 'jpg', :time => 10, :auto_rotate => true }
},
:processors => [:transcoder]
validates_attachment :file, :content_type => { :content_type => ["video/x-flv", "video/mp4", "video/ogg", "video/webm", "video/x-ms-wmv", "video/x-msvideo", "video/quicktime", "video/3gpp"] }
process_in_background :file
end
end
Hopefully this will save somebody else all the trouble.
I have an application which works already (in staging and prod) with S3.
Now we want it to work with cloudfront.
I figured out that from some reason I have paperclip definitions in two places:
/confog/initializers/paperclip.rb:
if Rails.env.production? || Rails.env.staging? || true
Paperclip::Attachment.default_options[:url] = ':s3_domain_url'
Paperclip::Attachment.default_options[:path] = '/:class/:attachment/:id_partition/:style/:filename'
end
/config/environments/staging.rb and /config/environments/production.rb
config.paperclip_defaults = {
:storage => :s3,
:s3_credentials => {
:bucket => s3_options[:bucket],
:access_key_id => s3_options[:access_key_id],
:secret_access_key => s3_options[:secret_access_key]
}
}
(I load s3_options from s3.yml file which I have)
First question - is it necessary (or on the other hand - is it wrong) to have these two places with configuration?
With this configuration I get this:
> Profile.last.image.url
=> "https://mybucket.s3.amazonaws.com/profiles/images/000/000/001/original/someimage.jpg?1439912576"
My goal:
Get cloundfront url instead of s3.
I tried several things:
Add to paperclip.rb this line:
Paperclip::Attachment.default_options[:s3_host_alias] = "xxxxx.cloudfront.net"
(where xxxxx stands for the cloudfront hash).
Result: nothing is changed.
Add to paperclip.rb this line:
Paperclip::Attachment.default_options[:s3_host_name] = "xxxxx.cloudfront.net"
(where xxxxx stands for the cloudfront hash).
Result: paperclip concatenate the bucket name before it:
> Profile.last.image.url
=> "https://mybucket.xxxxx.cloudfront.net/profiles/images/000/000/001/original/someimage.jpg?1439912576"
Disable configuration in paperclip.rb and add these lines to the environment config file (I tried it on development.rb):
config.paperclip_defaults = {
:
:s3_credentials => {
:
:
:url => "xxxxx.cloudfront.net",
:s3_host_name => "xxxxx.cloudfront.net",
:path => '/:class/:attachment/:id_partition/:style/:filename',
}
}
Result: paperclip concatenate the bucket name after it:
> Profile.last.image.url
=> "https://xxxxx.cloudfront.net/mybucket/profiles/images/000/000/001/original/someimage.jpg?1439912576"
As (3), but add these lines one level higher:
config.paperclip_defaults = {
:storage => :s3,
:url => "xxxxx.cloudfront.net",
:s3_host_name => "xxxxx.cloudfront.net",
:path => '/:class/:attachment/:id_partition/:style/:filename',
:s3_credentials => {
:
:
}
}
Result: Same as (3).
Briefly, no matter what I put in :s3_host_name, paperclip concatenate the bucket name in some place.
Some idea?
It was easier than I thought.
Looks like paperclip uses :url either as a string or as a reference for a symbol which indicates how to construct the url.
In my /config/environments/staging.rb and /config/environments/production.rb files I have now:
config.paperclip_defaults = {
:storage => :s3,
:url => ':s3_alias_url',
:s3_host_alias => "xxxxx.cloudfront.net",
:path => '/:class/:attachment/:id_partition/:style/:filename',
:s3_credentials => {
:
:
}
}
Ended using:
amazon = AppConfiguration.for :amazon
config.paperclip_defaults = {
storage: :s3,
url: ':s3_alias_url',
s3_host_alias: amazon.cloudfront,
path: '/:class/:attachment/:id_partition/:style/:filename',
s3_protocol: :https,
s3_credentials: {
bucket: amazon.s3_bucket_name,
access_key_id: amazon.aws_access_key_id,
secret_access_key: amazon.aws_secret_access_key
}
}
Alternative to AppConfiguration
config.paperclip_defaults = {
storage: :s3,
url: ':s3_alias_url',
s3_host_alias: ENV['AMAZON_CLOUDFRONT'],
path: '/:class/:attachment/:id_partition/:style/:filename',
s3_protocol: :https,
s3_credentials: {
bucket: ENV['AMAZON_S3_BUCKET_NAME'],
access_key_id: ENV['AMAZON_AWS_ACCESS_KEY_ID'],
secret_access_key: ENV['AMAZON_AWS_SECRET_ACCESS_KEY']
}
}
Important to say, don't use Rails.application.secrets here. It's not available when the config files are loading on rails 4.1.8 at least
I get the following error ArgumentError, missing required :bucket option
Looks like an error where Paperclip can't load the url because it can't find the :bucket name
I had my s3 credentials hardcoded for the development environment. Everything worked fine. To make the application more secure I moved it.
config\initializers\dev_config.rb
ENV.update YAML.load_file("#{Rails.root}/config/dev_vars.yml")[Rails.env]
config\dev_vars.yml
development:
S3_BUCKET_NAME: "####"
AWS_ACCESS_KEY_ID: "#####"
AWS_SECRET_ACCESS_KEY: "####"
config/environments/development.rb
config.paperclip_defaults = {
:storage => :s3,
:s3_credentials => {
:bucket => ENV['S3_BUCKET_NAME'],
:access_key_id => ENV['AWS_ACCESS_KEY_ID'],
:secret_access_key => ENV['AWS_SECRET_ACCESS_KEY']
}
}
I can't seem to figure out what is wrong
I've checked the YAML file is properly formatted
Running ENV['S3_BUCKET_NAME'] in the console gives the correct name
Can anyone help?
EDIT -
What I have done is hardcoded just the bucket name :bucket => "name" And that has worked, however I do not want to hardcode s3 information for security reasons.
Is there a reason the other variables are loaded but not the bucket name?
I followed the instructions from this link: https://devcenter.heroku.com/articles/paperclip-s3
And am getting the following error message:
ActionView::Template::Error (missing required :bucket option):
and it points to the line:
<%= image_tag company.profile_image.url %>
These are what my current files look like
env.rb:
ENV['S3_BUCKET_NAME'] = 'goodlifecompanyimages'
ENV['AWS_ACCESS_KEY_ID'] = 'XXX'
ENV['AWS_SECRET_ACCESS_KEY'] = 'XXX'
development.rb:
Paperclip.options[:command_path] = "/usr/local/bin/"
config.paperclip_defaults = {
:storage => :s3,
:s3_credentials => {
:bucket => ENV['S3_BUCKET_NAME'],
:access_key_id => ENV['AWS_ACCESS_KEY_ID'],
:secret_access_key => ENV['AWS_SECRET_ACCESS_KEY']
}
}
Thank you!
I'm having a similar problem to this issue, except I'm unable to modify the path at all.
I'm using Spree 2.2 which uses paperclip 3.4.2
I'm trying to modify the paperclip defaults to change the image path. All of the other configuration is working.
Myapp::Application.configure do
config.paperclip_defaults = {
:storage => :s3,
:s3_protocol => 'https',
:s3_host_name => "s3-eu-west-1.amazonaws.com",
:path => ":rails_root/public/:attachment/:id/:style/:basename.:extension",
:url => "/:attachment/:id/:style/:basename.:extension",
:s3_credentials => {
:bucket => 'xxx',
:access_key_id => "xxx",
:secret_access_key => "xxx"
}
}
end
But with this code the URL is like so:
https://s3-eu-west-1.amazonaws.com/bucketname/home/username/path/to/project/public/spree/products/24/original/ror_baseball.jpeg?1390110939
I've tried adding the following to that config block:
config.attachment_path = '/spree/products/:id/:style/:basename.:extension'
I've also tried adding the following to the config/initializers/paperclip.rb
Paperclip::Attachment.default_options[:url] = "/:attachment/:id/:style/:basename.:extension"
Paperclip::Attachment.default_options[:path] = ":rails_root/public/:attachment/:id/:style/:basename.:extension"
And also tried:
Paperclip::Attachment.default_options.merge!(
:path => ":rails_root/public/:attachment/:id/:style/:basename.:extension",
:url => "/:attachment/:id/:style/:basename.:extension"
)
Any ideas?
UPDATE: opened ticket on github
For Spree 2.2 (not 2.1), use the following code in initializers/spree.rb:
Spree::Image.attachment_definitions[:attachment][:url] = ':path'
Spree::Image.attachment_definitions[:attachment][:path] = '/spree/products/:id/:style/:basename.:extension'
The rest of the configuration should be set as 2.1 (see here for details).
I believe the /home/username/path/to/project/ part is being added by the :rails_root portion of your :path configuration.
If you look at the source code at:
https://github.com/thoughtbot/paperclip/blob/master/lib/paperclip/attachment.rb
You will see that the default values are as follows:
:path => ":rails_root/public:url",
:url => "/system/:class/:attachment/:id_partition/:style/:filename",
What if you set the following:
:path => ":url",
:url => "/spree/products/:id/:style/:basename.:extension",
Let me know what these do for you and we can see what else you need to do in order to get this working properly.
After a quick Google search, prompted by your comment I figured it out. I believe the answer is here, in the Spree documentation:
http://guides.spreecommerce.com/user/configuring_images.html
UPDATE: Apparently This is only available on older versions. In newer versions, it looks like you need to override Spree::Image. This is defined in vendor/bundle/gems/spree_core-2.2.0/app/models/spree/image.rb.