Delayed_Paperclip + Sidekiq + Mongoid-Paperclip - ruby-on-rails-4

Question

I'm working on an Rails 4.1 engine that handles user uploads of photos and videos. I'm using Mongoid-Paperclip to handle the uploads and Paperclip-av-transcoder to encode the videos into several formats. All files are stored at S3. All of that works fine, but as you can expect, encoding the videos can take quite some time, so the next step is to make that happen in the background. I did some googling and found Delayed_Paperclip that seems to do what I need. After that it seemed that Sidekiq was the best option for handling the background processing.
Now the problem is, I can't make all this work together. Running my unit tests I get NoMethodError: undefined method 'process_in_background' so it seems the problem resides on Delayed_Paperclip, although there is no special setup to it.
This is the model firing the problem
module MyEngine
class Video
include Mongoid::Document
include Mongoid::Paperclip
has_mongoid_attached_file :file,
:path => ':hash.:extension',
:hash_secret => "the-secret",
:storage => :s3,
:url => ':s3_domain_url',
:s3_credentials => File.join(Rails.root, 'config', 's3.yml'),
:bucket => "my-bucket-#{Rails.env}",
:styles => {
:mp4 => { :format => 'mp4', :convert_options => { :output => { :vcodec => 'libx264', :acodec => 'copy' } } },
:ogg => { :format => 'ogg', :auto_rotate => true },
:webm => { :format => 'webm', :auto_rotate => true },
:thumb => { :geometry => "250x187#", :format => 'jpg', :time => 10, :auto_rotate => true }
},
:processors => [:transcoder]
validates_attachment :file, :content_type => { :content_type => ["video/x-flv", "video/mp4", "video/ogg", "video/webm", "video/x-ms-wmv", "video/x-msvideo", "video/quicktime", "video/3gpp"] }
process_in_background :file
end
end
I've tried adding require "delayed_paperclip" to the lib/myengine/myengine.rb file but that didn't help.
Regarding Sidekiq, I have added to the test_helper.rb the following:
require 'sidekiq/testing'
Sidekiq::Testing.inline!
Note that I did not forget to run bundle install and Redis is up and running. I'm using Mongoid, not active record.
What I am doing wrong? Has anyone successfully used this setup? Is there another combination of gems that I should try?
Aditional info:
Delayed_paperclip 2.9.1
Mongoid 4.0.2
Mongoid-Paperclip 0.0.9
Paperclip 4.2.1
Paperclip-av-transcoder 0.6.4
Rails 4.1.9
Sidekiq 3.5.0

Answer 1

I've been digging through the code of delayed_paperclip and it is definitely tied to ActiveRecord so not compatible with Mongoid. I gave a try to mongoid_paperclip_queue but that gem hasn't been updated in 4 years and doesn't seem to work with the current versions of rails/mongoid/paperclip as far as I can tell.
I therefore decided the best way to solve my issue would be to override the code of delayed_paperclip that integrates with ActiveRecord and make it instead work with Mongoid.
This is what I ended up doing, and seems to be working fine so far:
lib/myengine.rb
require "mongoid_paperclip"
require "paperclip/av/transcoder"
require "delayed_paperclip"
require "myengine/engine"
module Myengine
end
DelayedPaperclip::Railtie.class_eval do
initializer 'delayed_paperclip.insert_into_mongoid' do |app|
ActiveSupport.on_load :mongoid do
DelayedPaperclip::Railtie.insert
end
if app.config.respond_to?(:delayed_paperclip_defaults)
DelayedPaperclip.options.merge!(app.config.delayed_paperclip_defaults)
end
end
# Attachment and URL Generator extends Paperclip
def self.insert
Paperclip::Attachment.send(:include, DelayedPaperclip::Attachment)
Paperclip::UrlGenerator.send(:include, DelayedPaperclip::UrlGenerator)
end
end
DelayedPaperclip::InstanceMethods.class_eval do
def enqueue_post_processing_for name
DelayedPaperclip.enqueue(self.class.name, read_attribute(:id).to_s, name.to_sym)
end
end
Then all you need is to include the delayed_paperclip glue to the model:
module Myengine
class Video
include Mongoid::Document
include Mongoid::Paperclip
include DelayedPaperclip::Glue # <---- Include this
has_mongoid_attached_file :file,
:path => ':hash.:extension',
:hash_secret => "the-secret",
:storage => :s3,
:url => ':s3_domain_url',
:s3_credentials => File.join(Rails.root, 'config', 's3.yml'),
:bucket => "my-bucket-#{Rails.env}",
:styles => {
:mp4 => { :format => 'mp4', :convert_options => { :output => { :vcodec => 'libx264', :acodec => 'copy' } } },
:ogg => { :format => 'ogg', :auto_rotate => true },
:webm => { :format => 'webm', :auto_rotate => true },
:thumb => { :geometry => "250x187#", :format => 'jpg', :time => 10, :auto_rotate => true }
},
:processors => [:transcoder]
validates_attachment :file, :content_type => { :content_type => ["video/x-flv", "video/mp4", "video/ogg", "video/webm", "video/x-ms-wmv", "video/x-msvideo", "video/quicktime", "video/3gpp"] }
process_in_background :file
end
end
Hopefully this will save somebody else all the trouble.