Last Wednesday (March 4th), I gave a two-part presentation to my fellow EverTrue employees on using ActiveRecord’s powerful migration management to handle non-Rails database schema changes and how complex queries can be built using Arel, along with how the responsibilities of ActiveRecord and Arel compare conceptually and practically.

A good part of the company attended (including some marketing, sales, and other non-technical folks), in no small part due to the free taco lunch buffet we were provided (and of course the prospect of an interesting presentation!). The diversity in the audience gave an inherently technical presentation a more widespread appeal, which prompted questions like “How do these changes get surfaced on the front end?”, and “Are our development processes less error-prone because of these changes?”. This sort of exposure helps inform the company and reinforce confidence in our engineers through examples and discussion. It felt good to give this kind of insight into our development organization and how we operate.

We recently started encouraging more presentations of this format, giving engineers a chance to share their experiences and expertise with their peers. It’s a win for everyone; organizing and presenting a topic helps the presenter clarify and hone their knowledge in their own minds while encouraging learning and discussion about development topics amongst coworkers. With a lot of development split between Java, Rails, and front-end technologies like React and Objective-C a lot of information sharing tends to naturally become isolated within those channels. It isn’t a conscious decision; just an artifact of the associations we form in our everyday work. Presentations like this open up the floor to everyone at EverTrue and encourage the sharing of concepts and experiences applied in everyday programming challenges. The hope is to ultimately grow the development organization as a whole and break down some of those barriers that keep useful specialized knowledge from being more widely shared.

In my mind the possiblities here are endless. I started brainstorming other potential topics that I think would benefit everyone here at EverTrue, and here are just a few that came to mind:

• Useful functions/advanced querying in ElasticSearch
• Building an API from scratch
• Useful testing tools/scenarios
• IOS development 101
• Security challenges and general security awareness
• Applying statistics to our contact data
• 5 most important tools in Rails development
• The contacts data model
• The authenication process
• Continuous integration - how it works now and how it could be improved
• AWS 101
• Transitioning from MySQL to Cassandra
• Building React components
• Code editor tips and tricks
• and a million more

I’ve summarized my talk’s two main points below, and have made the presenation slides available at the bottom for your viewing pleasure. Enjoy!

Part 1 - Using ActiveRecord to manage database schemas in non-Rails projects

Part 1 of my talk was motivated by a glaring database lock issue that exposed a more important problem: in our Java APIs there was essentially no database schema management process. The best we could do was commit a schema.sql file that was often neglected and not super-intuitive to use. Schema changes were a manual process, where work was duplicated across local, staging, and production environments. This meant that any change to our schemas was prone to a configuration error, and even worse the history of our changes was completely lost.

Enter a handy project called ActiveRecord Standalone Migrations. It provided the tools necessary to move the ever-so-useful schema management capabilities of ActiveRecord to any project that needed them. For us, that was our Java APIs.

The change gave us many benefits:

• Schema history tracking
• A consistent format for explicitly stating changes that could be applied across multiple environments in exactly the same way
• Migrations that can be run on deploy, making it impossible to forget about them or leave out a migration on one environment
• If we ever lose our databases we have an easy and automated way to recreate the schema from scratch

Part 2 - A comparison of ActiveRecord and Arel, and how to build complex queries

At EverTrue we are no strangers to the benefits of ActiveRecord in our Rails APIs. But the benefits of a somewhat obscure query-building language called Arel were less well-known. Online documentation for Arel is poor, with few resources for working with the language or explaining why it is beneficial. In my research for the topic I came across an excellent talk by Twitter’s Cameron Dutro at RailsConf 2014: Advanced aRel: When ActiveRecord Just Isn’t Enough. It gave me the inspiration, and more importantly the resources, I needed to share the topic with my team and others.

The core difference between ActiveRecord and Arel can be summarized as follows:

• ActiveRecord deals with retrieving/inserting data, acting as a persistence layer (by storing objects in memory), and handling some domain logic in Rails apps
• Arel is used solely to build queries to then be passed to ActiveRecord for execution

This means that Arel knows very little about your models or your database. Arel provides a way to create queries that in many cases look nicer than traditional ActiveRecord queries while also allowing queries that are just impossible to do otherwise (like outer joins). It can also easily do query optimization by making use of the AST structure it maintains under the hood, essentially representing queries as nested trees composed of operators and values that can be quickly swapped out for other pieces if necessary.

Arel has many other benefits over ActiveRecord. It provides an abstraction over the underlying query language. It gives you the benefits of Ruby syntax checking for free, keeping you from having to constantly scan long string queries for errors or typos. It has chainable methods, allowing for dynamic and flexible query construction. In many cases, especially for long, complex queries, it can be easier to read. The .to_sql method allows you to double-check the generated query, to be sure it is giving you what you expect. And it fits more naturally with the object-oriented nature of Ruby.

We have already started using Arel to construct queries in our Rails APIs, and it is my hope that this presentation has opened the doors for Arel to power more and more queries here at EverTrue, especially as they begin to grow in complexity.

Presentation Slides