Redirects validator and exporter

Background

When building bespoke websites we frequently migrate sites from old versions. While migrating it’s very common that some pages get merged or deleted, or the URL structure changes. When we launch the new site we need a system in place that maps the old URL to the new URL, and this is achieved by adding “rewrite rules” in a server configuration file (for example an .htaccess file for Apache servers).

Example rewrite rule in an .htaccess file:

RewriteRule ^our-team/?$ /about-us/meet-the-team/ [R=301,NC,L]

Example pseudocode rewrite rule in an .htaccess file:

RewriteRule ^{OldURL}?$ {NewURL} [R={RedirectStatus},NC,L]

Usually, as part of the migration process, we get a set of mappings from the client and we turn this into a set of redirect rules for the .htaccess file of the site. The easiest way for clients to provide these mappings is as an Excel spreadsheet.

The last time we did a site migration we had around 300 mappings but on average we get sent around 500 redirects per site. So could you imagine trying to add these rewrite rules for hundreds or thousands of pages? This process can be really repetitive and super tedious, even with the help of Regular Expressions to turn the mappings into the necessary format.

Errors within the initial mappings, for example, duplicate, invalid or conflicting redirects, are our main pain point. To remove these errors required someone to manually check through each of the redirects, which was a tough job for a human-looking through hundreds and keeping track of all previous rules!

What was developed?

In the office, we had discussed how helpful it would be if we had a tool to do this instead of manually checking and formatting.

So I put the initial idea into action and decided to design and develop a tool for validation and exporting redirects. This tool should take the mappings as a CSV file, validate them, and export valid mappings in the correct format for a .htaccess file. It should also output error and warning messages for any invalid or conflicting rules.

This tool could then be used internally to speed things up and reduce the likelihood of any errors creeping into the re-launch. The intention was also to share this tool for public use by the developer community.

First usage

As I finished developing my prototype we were in the process of migrating the Pavilion Publishing site, so it was the perfect opportunity to test out my new tool.

The client had provided us with 868 redirects and, after validation, there were 856 valid redirects exported. So there were 12 invalid redirects which more than likely wouldn’t have been picked up during visual inspection.

Example error messages returned:

This first validation attempt was done without “External checks” enabled and finished in about 4 seconds, which seems perfectly acceptable. “External checks” is a feature I added to request each “From URL”, and confirm that the request is redirected to the “To URL”. With this feature enabled, it took about 60 seconds to complete. Again this seems fine given the hundreds of request – response cycles required.

So my tool worked – we were able to report the invalid rules to the client and get them corrected before the re-launch!

Summary

I believe this lab project has been successful as it has made migrating a live site much smoother – we were able to iron out all redirect problems before going live. We had also saved a lot of time as this tool completed the whole flow of taking the mappings, validating and exporting into rewrites rules for the .htaccess file within a few seconds.

The online tool is now live for public use at https://rewrite.to, so we will hopefully see you there to make your life easier with your next site relaunch. We’d love to know if you find it useful, and comments and new feature requests are welcomed! Please email us via [email protected]