How we implemented file uploading in our CMS

Posted in: Beta, Content Publisher, Development

"dude-wheres-my-file

The setup

We have built into our CMS the ability to attach files to our Publication content type. Here I'll go cover the steps and hurdles we went through in implementing this very common feature.

On our first pass we took the often sensible approach of keeping scope small. We decided to build the ability to attach one file. Then we would iterate towards multiple attachments in later sprints.

Our first decision early on was to pick Thoughtbot's Paperclip library to help us implement this feature. Thoughtbot have a strong reputation among the Ruby community and we admire their approach of keeping things simple.

As part of our first iteration we had to do a ton of work around infrastructure set up too.

  • we needed dedicated network file storage for:
    • staging
    • training
    • and production
  • plumb in these file mounts to the CMS Editor so that we can show the file back to the user
  • changed our Mina deploy scripts to add an alias into RAILS_ROOT/public
  • and then copy the file on publish to the web server along with our HTML static page outputs

Simple!

The twist

After we shipped this first iteration we discovered a bug. Once you uploaded a file and hit save, if you encountered a Rails validation error you would lose your new attachment. So you'd fix your validation error but you wouldn't know that you had to upload that file again. This was especially bad if you were editing to change the attached file. If you had to fix other changes, you wouldn't realise that the attachment reverts to the original.

Twisting further

But Paperclip can handle this right? Wrong! It's a known bug (since 2009) that they won't fix! The most commonly suggested fix to this bug is to re-implement with an alternative library called CarrierWave.

A developer having a bit of a rage
Rage!

Act two: So CarrierWave...

...fixed this bug. It involved a rewrite of the code, but at least we could re-use all the infrastructure we'd set up. Next up, multiple attachments!

Another twist!

Our first attempt at implementing multiple attachments brought back the bug...

A developer having another rage
Double...
A developer having even more rage
...rage!

The final act

This time CarrierWave wasn't caching the attachment because we had applied it on the parent object in our polymorphic model. The fix was to re-do the code (again). Thankfully this time not as drastically as before. We are still using CarrierWave after all. We simply (!) applied CarrierWave to the Publication object itself rather than to its parent.

This model is closer to what we started with in our first implementation as well.

So now it's all simpler.

Still...

Despite all the hardship I was reminded that 10 years ago when we built the LMF we had to support file uploading. In that time we had to do a lot more of the work ourselves. Even though we had things like Apache Commons' IO library, all that gave us was binary streams and we still needed to write code to handle output to disk. That all took much, much longer than this rollercoaster attempt we made in Rails many years later. It's been an interesting contrast.

Posted in: Beta, Content Publisher, Development

Responses

  • (we won't publish this)

Write a response