Digital Marketing & Communications

We've seen 1s and 0s you wouldn't believe

Topic: Development

Automatically shipping early and often

📥  Beta, Development

We're an agile team, but rather than dogmatically follow a proscriptive capital-a Agile methodology we try to hew closely to the principles behind the Agile Manifesto.

Principle 3:

"Working software is delivered frequently (weeks rather than months)"

We're trying to do better than that. Sixty days after starting, we made the 100th release of our beta CMS to the staging server and the 21st release to our production server.

As Kelv has described before, "staging" is where features get deployed as and when they are completed so that we can run acceptance testing before they move into our production and training environments.

This little milestone represents many hundreds of individual changes by both developers and content writers over the last few months. Because each change is automatically sent to our servers when it's completed, this milestone happened completely transparently and with no fuss. This is exactly as it should be!

This is the first project where we've used Continuous Delivery. It did take some time to plan and set up the infrastructure in advance, and because we're still in development we're yet to battle-test the fact that the frequent deployments are totally seamless for people using the CMS. Nevertheless, the fact that we can deploy whenever we choose is a massive time-saver, allowing us to maintain a state of flow for sustained periods.

Some of our other projects are still using point-in-time pre-scheduled releases which we need to run manually and they already feel archaic. We're hopeful that we can take many of the processes we've developed during the CMS beta and apply them to these other applications, removing the burden of having to remember (or look up) many different deployment configuration intricacies.

 

Why static publishing and why Hugo?

📥  Beta, Development

As we've written before, we're going to try to use Hugo for statically publishing www.bath.ac.uk. There's a nice video showing it publishing 5,000 pages in under 7 seconds, and our testing gives similar numbers.

Static site serving has been big news over the past few years, mainly fuelled by the release of Jekyll. The problem for us is that we have many thousands of pages and many of these tools are written in Ruby or Node.js and are slow. If we want to change part of the site template then every page would need to be regenerated, and we want this to happen as quickly as possible.

At the moment there is almost no delay between making a change to the site and seeing it live on www.bath.ac.uk because we deliver pages straight from our database, but this means maintaining a more complicated and fragile infrastructure than we would like. Switching to a fast, statically published site will improve site stability and reduce our maintenance overhead, allowing us to spend more time on features.

Hugo is a young project and has some of the rough edges you might associate with that, but we're pretty hopeful that it will see us through the next period of innovation on the bath.ac.uk platform. If we do run into serious problems then we will probably switch to Jekyll until we were in a position to either scale out direct hosting from Rails or introduce a dedicated caching layer like Varnish.

 

A brief overview of the technical platform for alpha.bath.ac.uk

📥  Beta, Development

Our current CMS platform was introduced in 2007. It is a large, monolithic Java application which is hard to extend or customise. For the Alpha of a new CMS we wanted to base it on a popular web framework which would allow us to easily customise and extend a core platform, whilst also allowing us to use a broad range of third-party libraries.

We decided to base our Alpha on Rails, Contentful and Hugo. We used Rails to build a lightweight admin interface which changed our content in Contentful. Clicking 'publish' then sent a webhook notification to another Rails application we wrote which then used Hugo to regenerate the static HTML page. This general system design of loose coupling worked well and was a good introduction to the strengths and weaknesses of each of the tools we used.

We will keep Rails and Hugo as we move into the beta, but the libraries for interaction with Contentful were just too slow for us to use in real time. After we'd finished our Alpha a set of third-party libraries were released which might have helped, but nothing would be faster than connecting to a content store running here, and so that's what we'll do for the beta.

Deploying our Ruby on Rails applications

📥  Development, Tools

We are moving away from a Java based infrastructure to writing applications in Ruby on Rails. An important part of our former infrastructure was the ability to automatically deploy our software via the continuous integration server via a series of Ant tasks so we have been looking for a good way to do the same sort of thing with Rails code.

The established choice for Rails automated deployment seems to be Capistrano however there have been complaints about its speed and there seems to be a shift towards Mina which behaves differently under the hood.

Our intended development roadmap has us building a large number of small Rails projects all of which will need very similar deployment scripts so we also need a sensible way to manage the distribution of those scripts. We wrapped them in a gem which could simply be added to the project Gemfile and added a helper script which can set up a project and generate deployment new environments as needed.

We have made the gem public on our team github account with some instructions for use in the README and will be pushing updates as we add features and fix problems. If you find it useful, please let us know!

 

Using Rails

📥  Development

After ten years of Java and PHP, the Digital team have started writing all new applications, including our new CMS, in Ruby on Rails.

There are two reasons why now is a good time for us to do this. The first is that Rails is considered old hat, which means that it's now a tried and tested platform. The other is that improvements in our server virtualisation infrastructure mean that we no longer need to wait for Ruby packages to be available for our Solaris servers, because we can create new Linux servers as and when we need them. Solaris was a big blocker because the most recent Ruby package for Solaris is 1.8.7, which was released all the way back in 2008.

We are happy with our Java and PHP applications, and won't be looking to rewrite those in the near-term. We will continue to write applications in those languages where appropriate but are convinced that the Ruby community has shown itself to be strongly united behind Rails, making updates and community support much better than it is for the many competing Java and PHP frameworks.

Rails isn't perfect. It has regular security vulnerability announcements and then, assuming the patch is simple, we need to schedule the downtime to patch our applications. We have a channel in Slack which is subscribed to the rubyonrails-security mailing list and so we find out about new vulnerabilities quickly.

We're not currently planning on hosting public-facing applications with Rails, keeping them for internally-facing applications like the CMS editor. This will give us time to understand how to scale up our servers and applications appropriately and, since the CMS creates static HTML files it means we will be able to leave our powerful webservers doing the job they're already optimised for.

There's been a lot of learning involved in going from a blank-slate to a production-ready Rails application, and there'll be more to come, but we're happy with Rails and confident that it will serve us well in the coming years.

 

Our Github workflow

📥  Development

We are only 10 months into adopting Github Enterprise for our version control system and have many things still to learn. Near the start of that period we needed a workflow for managing the contributions that our team of devs, designers and editors make to our projects.

We decided to have a look at what was out there first and adapt something to our needs, taking into consideration our relative inexperience with Git in general.

Pick one

This helpful guide by Atlassian to some workflows was good at giving us an intial overview. We were able to discount the various workflows for either being to complicated for us (Gitflow) or too basic (Forking workflow).

We looked at the Github Workflow as well and found it suitable for our level of experience and how we wanted to work.

Change it

There are a couple of minor things we do differently though in our team. First of all we don't work against a branch called master.

What we have instead are staging and production branches. These reflect our platforms that we deploy to. staging is actually the default branch of all our project repos.

We always branch from staging to work on our features. Our Pull Requests are for merging back into staging. We then merge into production from staging. This last bit is still a bit clunky as we have to issue another PR for that merge.

Push it

Salt n Peppa - Push It

 

The staging branch for a project deploys automatically. Changes to repos send triggers from Github to Bamboo (using the built in Bamboo Service Hook) to run build and deploy plans. For production we'd rather have control over when this happens. So while builds are still automatic for production, we manually trigger deployments at the push of a button.

We are using mina to push our apps to our servers, executed by Bamboo. Bamboo has an official Ruby plugin which we are using. This includes running Bundler tasks.

More polish?

What we have so far is a little bit rough still on some edges (the PR to production for example), but overall we are finding that this workflow model is working well.

This is not to say that as we gain more experience we won't revisit the various models of workflow in the future.

 

Alpha site release notes, 12 - 19 January

📥  Beta, Communication, Development, Style, content and design

On Monday 12 January, we launched an alpha version of our site to internal users, including a new external homepage and the about section, which is controlled by our new CMS alpha. Collectively we've been referring to these as the Alpha site.

The Alpha site is still a work in progress, and we're testing it with our internal users now because your feedback will help to shape its development. We want to see how you use it, whether it meets your needs as a user, and what problems you encounter — so we can fix them before they wreak havoc on a larger audience.

What we did this week

We've had a lot of useful feedback from our users already, and we've fixed some of the issues you've raised, as well as continuing to tackle pre-planned work from the Alpha backlog.

  • Fixed issues that meant some maps were difficult to find or unavailable, including updating redirects from old URLs and changing which page included the map links.
  • The homepage is now served statically, which will lead to performance improvements and better stability if the CMS is unavailable.
  • Improved how the lead image on the homepage displays on smaller screens. A much larger proportion of the image is now visible and editors can select the alignment for how the image is cropped.
Screencap of the homepage Alpha.

Image cropping on small screens before.

Screencap of the homepage Alpha.

Image cropping on small screens after.

  • Worked on a first iteration of an editorial calendar for the homepage to ensure it's frequently updated with interesting, relevant content.
  • Replaced the National Student Survey graphic in the homepage footer with a better-looking SVG version.
  • Standardised design elements in the header and search bar across the Alpha site.
  • Reordered the link lists on the about landing page so it now leads with our mission and values.
  • Changed the wording that links to location pages on team overview pages from "The team is located in [location page]." to the more flexible "Find out more about this team's location in [location page]."

Keep it coming

Please continue to share your feedback with us. Your feedback is most helpful when it's specific - let us know what task you wanted to accomplish on the website and what stopped you, and we'll look into how we can fix it.

 

Building alpha.bath.ac.uk

📥  Beta, CMS, Development, Tools

We've changed just about everything to do with how we manage bath.ac.uk for the alpha, and this includes the underlying technology.

The live www.bath.ac.uk is hosted dynamically from a monolithic Java-based CMS (OpenCms) running on Solaris servers with Oracle as a back-end database and all the code is in Subversion.

The alpha site is being served from static HTML files generated by Hugo. The inputs are provided by multiple Ruby on Rails apps running on Linux servers using MySQL, MongoDB and an off-site service for data storage, and all the code is in our local GitHub Enterprise installation and is deployed from Bamboo by Mina.

The difference could not really be much bigger! It's certainly been a challenge to get up to speed with so many new things all at once but we think that this stack, or some version of it - that's what alphas are for! - should be more future-friendly than our existing infrastructure.

We'll go into more detail on how we're using each piece of technology in the following days and weeks.

Show & Tell, November 21 2014

📥  Communication, Design, Development, Show & Tell

Back in our spiritual home after the impromtu reshuffle that made our last Show & Tell session so special, we had a full roster of presenters and a diverse range of topics.

Ruby Idioms - Kelvin

Our developers are working more and more with Ruby — Rails in particular — and Kelvin has been challenged with providing instruction and direction to the team on the subtleties of how we should write Ruby differently from Java and PHP (other previous go-to production languages).

Far too much to cover in five minutes, we instead had a whistle-stop tour of the top ten seven things to be aware of, from not using unless statements with an else block, to replacing do...end blocks with curly braces if they are a single line.

A full rundown can be found in Kelv's github repo: https://github.bath.ac.uk/mnskchg/ruby_idioms_show_and_tell.

Less stuff - Dan

Still with me? Excellent. Next up was Dan, who talked us through taking a pragmatic approach to webfonts to provide a better user experience. A large part of the work was reducing the filesize of the font manifest file by 66% - theoretically providing a significantly improved loading time for those viewing the website on slow internet connections. The key was looking at the different weights of font being served by default, and making careful design choices that allowed us to provide maximum clarity and aesthetic with the minimum variety of styles and weights.

Dan then went on to propose a manifesto of using less as a starting point for design - tying in aspects of user-centered design, progressive enhancement, the mobile-first approach, and our existing delivery principles.

What do I do? - Katrina

Six months into her new post as Research Marketing Manager, guest speaker Katrina gave us the lowdown on what her job entails. It turns out that a fair amount of it is commercially sensitive, so I'll be skipping over that - no secrets for you.

Katrina spends a large amount of her time planning and coordinating large-scale campaigns to cement relations with University stakeholders. Currently we are tapping into the large amount of water-themed research that our academics are involved in and Katrina is putting the finishing touches to a six-month campaign relating to this.

When not devising ways to get our research the recognition it deserves, Katrina acts as a single point of contact between our academics and the various marketing teams that exist on campus at all different levels - from research teams, through departmental and faculty right up to the University Marketing and Comms. This aspect of her role has been extremely well-received on campus, as busy professors delight in having one single consistent person to deal with concerning their marketing.

The tale of BrowserStack - Tom

Continuing his series of talks concerning security, Tom Natt used the real world example of the recent attack on BrowserStack to illustrate what can happen when things go wrong.

Essentially, BrowserStack had an old computer that nobody used or maintained but was still connected to their network. A hacker discovered this and used the Shellshock vulnerability to take control and gain access to the API key for their AWS (Amazon Web Storage). From this they discovered the database password and attempted to download their entire customer database. This was when BrowserStack became aware of the hack and acted quickly to shut them down. It is still reckoned that 1% (approximately 5000 users) of the database was compromised.

We were about to use BrowserStack to assist us with some work, so this attack and the way that BrowserStack handled it (in terms of securing their system and managing their public profile) went a long way to reassuring us that they were still a suitable partner. Tom also made the point that having just been hacked, they were likely to be awake to the danger right now because of their recent experiences.

Alpha update - Ross

Our the last seven weeks the team has been working on several alphas (CMS, homepage, events, prospectus) and these have all been presented to members of our Digital Steering Group - which is comprised of almost all of our pro-vice-chancellors as well as the movers and shakers in senior management. The feedback has been overwhelmingly positive, with very enthusiastic engagement with aspects of the homepage and the CMS in particular (our most 'mature' alphas). Our plan was always to get the new homepage and areas of the site controlled by the new CMS in front of staff and students as soon as possible, and with the full support of the DSG, we are looking to do this in December.

 

Show & Tell, October 24 2014

📥  CMS, Design, Development, Show & Tell

We celebrated 11 months of Show & Tell last week with another Show & Tell. On the agenda this week: editorial calendars, rebasing, analytics and some exciting new alphas.

Flow - Charlotte

Last month, Rhian talked about how she and Charlotte had searched for an editorial calendar tool that did everything they needed, but didn't come packed with superfluous features. After initially turning to Google Spreadsheets, they've now spent two weeks trialling Flow, and Charlotte updated us on their first impressions.

They particularly liked how Flow offers:

  • a clear calendar view, with the option to drill down into the details
  • quick scheduling and tagging of items
  • dragging and dropping items between dates
  • the option to look at individual calendars, or an overview of everything.

Overall, we like Flow and we might use it for more editorial calendars in the future.

Ace of Rebase - Tom N

We've been using GitHub more and more for our version control. Mostly this has been a happy and prosperous time, but one aspect of Git has been repeatedly responsible for screams of horror in the office: rebasing. To provide some context for the screams, Tom talked us through what rebasing is, why you should do it and what can go wrong.

Git allows multiple people to work on a project simultaneously and engage in jolly cooperation. One master copy of the project exists, with everyone working on different features in their own individual branches, which are copied from the master. You can then merge these branches back into the master copy and combine everyone's work.

But what if other people have merged their branches into the master copy before you? Your branch is now out of date. You may know the feature you've been working on will still play nice with the new master, but if you try to merge it back in normally, you might overwrite work that people have done since.

This is where rebasing comes in. Essentially you need to change history and pretend your branch started with the current master, not the one you actually started with several versions ago. You can then add your work to the project without wiping out anything else that's been done since you started it.

Boromir: "One does not simply Git rebase it!"

Boromir, who destroyed the fellowship of the sprint team after overwriting the repository.

Rebasing is fiddly stuff - when it's done right, it's great, but get it wrong and you could lose a lot of work on a project. Tom's top tip for avoiding extreme pain and suffering was to use interactive mode when you rebase, or everything goes boom.

bath.ac.uk analysis - Takashi

Are we putting the right amount of effort into the right places? Takashi recently did a top level analysis of bath.ac.uk to compare the size of our various sections with how much attention they get from our users. He presented his findings, which included:

  • our study section gets the most traffic
  • the Computing Services section is our biggest in terms of size and number of files
  • many CMS users have edited less than ten pages in the last six months, but some have edited over 500
  • the rate of users accessing bath.ac.uk from Google is rising, while traffic from Facebook is dropping
  • visits from social media tend to be shorter and less engaging.

Takashi also wowed the room with some good-looking treemaps and fielded some questions about his information visualisation secrets (Google Spreadsheets).

Prospectus alpha - Rhian and Liam

Rhian, Liam and Tom T have spent several weeks working on a new app to collect and update the information in our prospectus.

We want our prospective students to get a consistent experience, so we need to integrate the content we put on the web and print versions of our prospectus.

Rhian and Liam gave us a demo of the shiny new Ruby app and showed us how adding a new course would work. User testing has already given the team some very useful feedback, so they'll be testing it with more of the app's future users as they continue to work on it.

Future iterations will tackle workflow and version control, and we hope the new app will be an improvement for the people who create the prospectus and the people who read it.

CMS alpha - Dan

The CMS alpha is the biggest project we're working on at the moment, and Dan's been handling the user interface (UI) for the editing app.

The UI for the CMS needs to be:

  • clear
  • concise
  • consistent
  • accessible.

After setting out these requirements, Dan gave us all a demo of the UI he's been working on and showed us what editing different content types will look like.

Ros from Monsters Inc with "FORMS" in big letters

Sometimes you've just gotta have them.

The new UI looks much more straightforward and user-friendly than our current CMS, and it also confirms to Level AA of the Web Content Accessibility Standards.

Work on the alpha continues this week. If you want to find out more, have a look at Ross's blog post about our plans for the CMS.