A big part of a technical SEO’s job is managing URL redirects. Whether we like it or not — because inbound links are often lost when URLs are redirected — URLs change. What’s just as important as deciding when a URL moves is how a URL moves.
There are myriad tools you can use to help with this process (Google Search Console is a go-to), but a crawler like Screaming Frog or a Saas like Ahrefs or SEMrush can also help. I’m an old school, manual, hands-on kind of SEO, so while I do check GSC for accurate data about which URLs are (or aren’t) getting traffic, I use these other tools instead.
Httpstatus.io
Definitely one of my Top 5 favorite tools, this dead-simple and completely accurate tool lets you upload a list of URLs and then check if they redirect (and how and where), if they return 200s, 4xx, or 5xx errors. The free version lets you do 100 URLs at a time.
You can even set a wide range of user agents to test with. You can see in this image that you can from several search engine crawlers, browser types for both desktop and mobile, and a few social media bots, too.
Quick caveat: if you have an actively managed firewall you may need to whitelist the tool. I use Cloudflare and I set up my rule using both the user agent (httpstatus.io) and the IP address (188.166.109.207).
Get started with checking response codes, headers, and redirect chains in bulk with httpstatus.io.
htaccess tester
Let’s face it, .htaccess can be hard to manage. Especially when you have multiple rules in play at any given time. .htaccess is the first stop for any incoming request for an Apache web server (after you get through the firewall).
Every single request for a website — images, locally hosted fonts, JS, and CSS — hits the .htaccess file. If your file gets too big and bulky, it could slow down your Time to First Byte (TTFB) and affect your overall page speed. And let’s face it, poorly managed sites often have big and bulky .htaccess files, mostly because of too many prior redirects.
It’s often important to use Regular Expressions (the next tool on my list) to set up a single redirect that matches a pattern instead of a list of individual URLs with individual destinations. But because these rules aren’t easy for most mortals to read, it’s easy to create conflicts, redirect chains, or just break the .htaccess file altogether with poor syntax.
Enter htaccess tester by made with love. This tool lets you upload your existing .htaccess file (you’ll need to delete all of the rules that aren’t rewrite rules), put in a URL that should be manipulated by .htaccess rules, and then see both if it goes where you want it to and which specific rules are triggered. This tool changed my .htaccess life from one of torture to one of normal-amount-of-frustration-with-coding.
Regular Expressions 101
Nothing strikes fear into the heart of a technical SEO like regular expressions. If they were being honest, most programmers would tell you they loathe them, too.
Regular expressions are an incredibly powerful pattern matching tool (see earlier comment about their use in .htaccess files). There are different syntax standards and even versions of regex (the standard shorthand for Regular Expressions) and its important to know the rules for the tool you’re using.
For example, you can use regex in GSC (they use RE2 syntax) to match groups of pages or queries that don’t necessarily follow the exact same pattern. Using the GSC regex matching tool is a great way to compare your website’s branded traffic (including typos) vs non-branded traffic.
Branded traffic queries (the | [pipe] is an
OR operator)
Use a wildcard operator to find all of the words which start with cash and end with solution
Everything else
Or use it to compare two sets of URLs
With the brilliant tool at RegEx101.com you can choose your “flavor” of regular expressions, offer up a string or paragraph to test against, and then start writing. The tool has a build-in cheatsheet, explains when things are working or not, and provides some information about how to resolve it.
While this tool is certainly indispensable, it is not without its frustrations. Those frustrations stem from the user’s lack of understanding RegEx, not the tool’s malfunctioning. In setting up this next example, I had to fumfer around a bit before I was able to accomplish what I wanted to demonstrate (which included consulting ChatGPT, a not-flawless coding assistant).
Below the screencap from RegEx101.com, I show the use of that in GSC. You can see it returns queries that match both sauvignon and sauv. GSC’s regex implementation is naturally case-insensitive, so you don’t need the piped expression I provided. .htaccess is definitely case sensitive and you need to explicitly tell it when you want a case-insensitive match.
