Check SSL certificates, security headers and redirects for a whole domain list

Checking a single domain's security basics means three tools: openssl s_client for the certificate, curl -I for headers, and a manual read of robots.txt. Doing it for 40 client domains, 300 sales prospects or your own portfolio of brand sites is a day of copy-paste. This guide shows how to run all of those checks over a list of domains in one go and get one row per domain with a 0–100 securityScore, an A–F securityGrade and plain-English issues — for $5 per 1,000 results.

What you get

Each domain returns 40+ fields across four areas:

A real row from a recorded run:

{
  "domain": "sqlite.org",
  "certValidFrom": "2026-08-01T01:03:35.000Z",
  "certValidTo": "2026-10-30T01:03:34.000Z",
  "certDaysUntilExpiry": 11,
  "certIssuerOrg": "Let's Encrypt",
  "certIssuerCN": "YE1",
  "certSubjectCN": "a1.sqlite.org",
  "certSanCount": 29,
  "certAuthorized": true,
  "tlsProtocol": "TLSv1.3",
  "finalUrl": "https://sqlite.org/",
  "redirectHops": 1,
  "redirectChain": [
    {
      "url": "http://sqlite.org/",
      "status": 301,
      "location": "https://sqlite.org/"
    },
    {
      "url": "https://sqlite.org/",
      "status": 200,
      "location": null
    }
  ],
  "httpsRedirect": true,
  "wwwRedirect": false,
  "hasHsts": false,
  "hstsMaxAge": null,
  "hstsIncludeSubdomains": false,
  "hstsPreload": false,
  "hasCsp": false,
  "cspHasUnsafeInline": false,
  "xFrameOptions": null,
  "xContentTypeOptions": null,
  "referrerPolicy": null,
  "permissionsPolicy": null,
  "server": null,
  "xPoweredBy": null,
  "cacheControl": "max-age=120",
  "cookieCount": 0,
  "secureCookieCount": 0,
  "httpOnlyCookieCount": 0,
  "sameSiteCookieCount": 0,
  "robotsStatus": 200,
  "robotsDisallowAll": false,
  "robotsSitemapCount": 0,
  "aiBotsDisallowed": [],
  "llmsTxtExists": false,
  "llmsTxtTitle": null,
  "llmsTxtSize": null,
  "securityScore": 50,
  "securityGrade": "C",
  "issues": [
    "TLS certificate expires in 11 day(s)",
    "No HSTS header",
    "No Content-Security-Policy header"
  ],
  "error": null,
  "scrapedAt": "2026-10-18T17:00:59.536Z"
}

The certificate data comes from a direct TLS handshake on port 443; the rest is ordinary HTTP requests — the same public information any browser receives when it connects.

How the score works

Points are earned for: HTTPS redirect (+20), a valid, trusted certificate (+20) with more than 14 days left (+5), HSTS (+15), Content-Security-Policy (+10), X-Frame-Options, X-Content-Type-Options, Referrer-Policy and Permissions-Policy (+5 each), no X-Powered-By leak (+5) and all cookies carrying the Secure flag (+5). Grades: A ≥ 85, B ≥ 70, C ≥ 50, D ≥ 30, otherwise F. Every missed point is spelled out in issues, e.g. "No HSTS header" or "TLS certificate expires in 9 day(s)".

Step-by-step

1. Open the tool and paste domains into Domains. Bare domains, full URLs and www. are all accepted — https://www.example.com/store is normalised to example.com.
2. Keep all four checks on for the full audit: Check TLS certificate, Check HTTP security headers, Follow redirect chain and Check robots.txt and llms.txt. Turn one off to skip that section.
3. Raise Max concurrency for long lists and click Start.
4. Use the ready-made Overview, TLS certificate and Headers, redirects & robots views, or export everything to CSV or Excel.

Run Domain Security Audit (TLS, HTTP headers, redirects, robots) on Apify →

Call it from code

import json
import os
from apify_client import ApifyClient
client = ApifyClient(os.environ["APIFY_TOKEN"])
run = client.actor("webdatatools/domain-security-audit").call(run_input=json.loads("{\n    "domains": [\n        "apify.com",\n        "example.com"\n    ],\n    "checkTls": true,\n    "checkHeaders": true\n}"))
for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(item)

import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: process.env.APIFY_TOKEN });
const run = await client.actor('webdatatools/domain-security-audit').call({
    "domains": [
        "apify.com",
        "example.com"
    ],
    "checkTls": true,
    "checkHeaders": true
});
const { items } = await client.dataset(run.defaultDatasetId).listItems();
console.log(items);

curl -X POST "https://api.apify.com/v2/acts/webdatatools~domain-security-audit/run-sync-get-dataset-items" 
-H "Authorization: Bearer $APIFY_TOKEN" 
-H "Content-Type: application/json" 
-d '{"domains":["apify.com","example.com"],"checkTls":true,"checkHeaders":true}'

Recipes

Certificate-expiry alerts for your domain portfolio

Schedule a daily run over every domain you own and add a webhook. Your endpoint filters rows where certDaysUntilExpiry is under 14 — or certAuthorized is false — and pings the on-call channel. An unreachable domain still returns a row with error set, so outages surface too.

Security hygiene report for clients

MSPs and consultants run the client's domains monthly and paste securityGrade and issues straight into the report. Because the scoring rules are fixed and published above, month-over-month changes mean something.

AI-crawler policy survey

aiBotsDisallowed lists which AI crawlers (GPTBot, ClaudeBot, Google-Extended, PerplexityBot and others) a site blocks in robots.txt, and llmsTxtExists shows whether it publishes an llms.txt. Run it over a competitor set to see who is opting in or out of AI search.

How much does it cost?

Primary event rates from a catalog snapshot; this is not the total price of a run. Additional charge events and Actor Start fees may apply; start fees can scale with memory. Check the Actor's Pricing tab on Apify for all events, conditions and current plan-specific rates, and set a maximum run budget before starting.

Each domain is one TLS handshake, up to 10 redirect hops, one headers fetch and two small text fetches, so platform usage stays negligible. Set Maximum cost per run in Apify and the tool trims the list to fit the budget.

How it compares

Monitor it every day

An audit is a snapshot. If you look after many domains, Domain Watch re-checks them every 24 hours and alerts you by email, webhook or Telegram before domain or TLS certificate expiry (30, 14, 7 and 1 days ahead), and when SPF, DMARC or nameservers change or a new hostname appears in certificate-transparency logs. To try one domain first, use the free instant domain check, no sign-up.

FAQ

Is this a penetration test?

No. It reads public, passive signals — the certificate, response headers, redirects and robots.txt — that any visitor receives. It doesn't probe for vulnerabilities, so it's safe to run against prospects and third parties.

What happens if a domain doesn't resolve?

You still get a row, with error explaining the failure and the other fields null. One dead domain never fails the whole run.

Why does a site with a valid certificate still score low?

The certificate is only a quarter of the score. Missing HSTS, no Content-Security-Policy or a leaking X-Powered-By header each cost points; issues lists exactly which.

Which headers matter most?

HSTS carries the biggest weight after HTTPS itself, followed by Content-Security-Policy. The smaller headers (X-Frame-Options, X-Content-Type-Options, Referrer-Policy, Permissions-Policy) are quick wins worth 5 points each.

Is it legal to check other people's domains?

Yes. Certificates, headers and robots.txt are public information servers send to anyone who connects, and no personal data is collected.